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Abstract 


Defence  Research  and  Development  Canada  (DRDC),  a  special  operating  agency  of  the 
Department  of  National  Defence,  contracted  Dalhousie  University  to  support  the  development  of 
a  management  accountability  framework  (MAP)  as  part  of  DRDC’s  EXPEDITION  09 
organizational  change  project.  The  DRDC  MAE  used  the  Treasury  Board  MAE  as  a  model,  with 
Expectations,  Indicators  and  Measures  tiers.  The  proposed  DRDC  MAP  reflects  the  DRDC 
vernacular  and  utilizes  a  survey  instrument  to  assess  managerial  cadre  performance  directly, 
rather  than  using  policies  and  standard  operating  procedures  as  proxies  for  organizational 
performance.  Pilot  studies  were  used  to  test  the  survey  instrument  approach  against  the  six  core 
elements  of  the  DRDC  MAP:  Policy  and  Programs;  People;  Service;  Risk  Management; 
Stewardship;  and  Accountability.  The  results  of  the  pilot  studies  confirm  the  feasibility  of  the 
survey-based  approach  and  provide  a  first  assessment  of  DRDC’s  organizational  performance 
against  indicators  for  the  elements  of  the  proposed  DRDC  MAP. 


Resume 


Recherche  et  developpement  pour  la  defense  Canada  (RDDC),  organisme  de  service  special  du 
ministere  de  la  Defense,  a  embauche  I’universite  Dalhousie  pour  P aider  a  elaborer  un  cadre  de 
responsabilisation  de  gestion  (CRG)  pour  EXPEDITION  09,  projet  de  changement 
organisationnel.  Ee  CRG  de  RDDC  s'inspire  du  CRG  du  Conseil  du  Tresor  comme  modele  et  de 
ses  paliers  Attentes,  Indicateurs  et  Mesures.  Le  CRG  de  RDDC  propose  reflete  les  termes 
communement  employes  a  RDDC  et  utilise  un  sondage  pour  evaluer  directement  le  rendement 
des  gestionnaires.  Sont  en  outre  examinees  les  politiques,  les  procedures  de  fonctionnement 
habituelles  et  Pinformation  de  Porganisme  comme  mesures  du  rendement  de  I’organisme.  Des 
etudes  pilotes  ont  ete  effectuees  pour  tester  I'approche  par  sondage  au  vu  des  six  elements  cles  du 
CRG  de  RDDC  :  Politique  et  programmes.  Employes,  Service,  Gestion  du  risque,  Gerance  et 
Responsabilisation.  Ee  resultat  des  etudes  pilotes  a  confirme  la  faisabilite  de  I’approche  par 
sondage  et  fourni  une  premiere  evaluation  du  rendement  organisationnel  de  RDDC  au  vu  des 
indicateurs  pour  les  elements  du  CRG  de  RDDC  propose. 
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Executive  summary 


A  DRDC  Management  Accountability  Framework:  Final  Report 

Judy  A.  Baroni;  Warren  C.E.  Nethercote;  DRDC  Atlantic  CR  2009-136;  Defence 
R&D  Canada  -  Atlantic;  September  2009. 

Introduction  or  background:  Defence  Research  and  Development  Canada  (DRDC),  a  special 
operating  agency  of  the  Department  of  National  Defence,  contracted  Dalhousie  University  to 
support  the  development  of  a  management  accountability  framework  (MAP)  as  part  of  DRDC’ s 
EXPEDITION  09  organizational  development  project.  The  proposed  DRDC  MAE  developed 
under  the  contract  uses  the  structure  of  the  Treasury  Board  (TB)  MAE  to  demonstrate  the 
relationship  between  the  two  frameworks.  Eike  the  TB  MAP,  the  DRDC  MAP  has  Expectations, 
Indicators,  and  Measures  tiers,  where: 

•  Expectations  demonstrate  leadership  standards; 

•  Indicators  convey  practical  meaning  of  the  Expectations;  and 

•  Measures  are  expressions  of  performance. 

The  proposed  DRDC  MAP  differs  from  the  TB  MAP  in  that  the  focal  point  for  the  DRDC  MAE 
is  the  DRDC  management  cadre,  rather  than  the  Deputy  Minister,  for  the  Treasury  Board  MAP. 

The  vision  of  the  proposed  DRDC  MAP  follows: 

The  DRDC  MAE  is  a  governance  tool  that  facilitates  dialogue  between  managers  and 
executive  leadership,  helping  to  clarify  managers’  understanding  of  their  roles  and 
responsibilities  and  providing  a  framework  for  assessment  and  improvement  of  business 
processes. 

Results:  The  Expectations  Tier  of  the  proposed  DRDC  MAP  was  first  derived  from  the  Treasury 
Board  MAP  through  consultation  with  the  DRDC  Deputy  Director  General  and  Corporate 
Services  Management  communities.  The  Expectations  tier  was  subject  to  further  minor 
modifications  during  subsequent  development,  and  reflects  the  vernacular  and  circumstances  of 
DRDC.  Por  example,  the  ‘Citizen-centred  Service’  element  of  the  Treasury  Board  MAP  became 
‘Service’  in  the  DRDC  MAP  because  neither  DND  nor  DRDC  focus  on  direct  services  to 
citizens. 

The  four  pillar  elements  that  form  the  perimeter  of  the  DRDC  MAP  set  the  environment  of 
DRDC  and  the  attitudes  and  behaviours  that  best  represent  the  organization.  The  six  core 
elements  in  the  centre  of  the  MAP  apply  across  the  organization,  but  are  especially  relevant  to 
middle  managers.  The  core  elements  set  the  standards  for  the  performance  of  management.  The 
pillar  elements  also  set  standards  for  the  performance  of  management,  but  more  at  the  strategic 
than  the  operational  level. 
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Governance 

and 

Strategic 

Direction 

The  DRDC 
Charter,  the 
Defence 
Science  and 
Technology 
Strategy  and 
the  DND 
Business 
Planning 
process  ensure 
effective 
support  and 
functional  S&T 
authority  for 
Government, 
the  Canadian 
Forces,  and 
defence  and 
security 
partners 


Defence  R&D  Canada  Agency  Values 

Through  its  actions,  DRDC  Leadership  continually  reinforces  the  importance  of  the  agency’s 
values  and  of  defence  ethics  in  the  delivery  of  results  to  Canadian  forces  and  defence  and 
security  partners. 


Policy  and  Programs 

DRDC  research  and 
analytic  capacity  is 
developed  and  sustained 
to  assure  high  quality 
policy  options,  program 
design  and  advice  to  DND/ 
CF,  Public  Safety  Canada, 
and  security  partners. 

People 

DRDC  has  the  people, 
work  environment, 
leadership  and  capacity  to 
provide  the  best  in  science 
and  technology  for  defence 
and  security,  now  and  for 
the  future 

Service 

S&T  activities  are  client 
and  partner-centred, 
policies  and  programs  are 
developed  to  address 
client  and  partner  needs. 

and  partnerships  are 
encouraged  and  effectively 
managed. 

Risk  Management 

DRDC  prepares  for, 
manages  and  exploits 
risk  in  the  science  and 
technology  environment. 

Stewardship 

The  DRDC  Control 
regime  (assets,  money, 
people,  services,  etc.)  is 
integrated  and  effective, 
and  its  underlying 
principles  are  clear  to  all 
staff. 

Accountability 

Accountabilities  for  results 
are  clearly  assigned  and 
are  consistent  with 
resources,  and  delegations 
are  consistent  with 
capabilities. 

Learning,  Innovation  and  Change  Management 


DRDC  fosters  creativity,  innovation,  and  change,  promotes  organizational  learning,  values 
corporate  knowledge,  and  learns  from  its  performance 


Results 

and 

Performance 


Relevant 
information  on 
results 
(internal, 
service  and 
program)  is 
gathered  and 
used  to  make 
decisions  in 
DRDC.  and 
agency  and 
public  reporting 
is  balanced, 
transparent, 
and  easy  to 
understand. 
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Governance 

and 

Strategic 

Direction 


•Support  to  DM 
DND  through  ADM 
(S&T) 

•Management 
framework  aligned 
to  strategic 
outcomes 

•The  right 
executive  team 

•Results-focused 
corporate  priorities 

•Strategic 

resource 

ailocation/reaiiocat 
ion  based  on 
performance 

•integrated 

management 

agenda 

•Horizontal 

collaboration 

•Environmental 

scanning 


Defence  R&D  Canada  Agency  Values 

•Exemplary  conduct 

•Agency  values  are  reflected  by  the  realities  and  culture  of  its  Centres 
•Values-based  management  practices 


Policy  and  Programs 

•Sustained  analytic  capacity  and 
culture  of  consultation,  review  and 
challenge 

•Results  focused  policy  and  program 
agendas  aligned  with  government’s 
horizontal  priorities 
•Social  Evaluation  of  Impacts 
•Confidence  of  the  ADM  (S&T), 
clients  and  partners 


People 

•Workforce  is  sufficient  in  numbers 
and  abilities  while  reflecting  the 
Canadian  population 
•Workplace  is  a  supportive  and 
respectful  environment,  including 
respect  for  official  language 
requirements 
•All  levels  of  management 
demonstrate  effective  leadership 
•Employees  are  engaged  In  their  work 
and  have  opportunities  to  learn  and 
grow 


•Proactive  engagement  of  clients 
and  partners 


Risk  Management 


•Risk  Management  is  integrated  with 
planning  and  operations  structures 
throughout  all  levels  of  the  Agency 
•Capacity  exists  to  support  a  Risk 
Management  Culture 


•Management  systems  provide 
relevant  information  and  effective 
feedback  on  resources,  results  and 
controls 

•Effective,  efficient  and  economical 
project  management 
•Established  internal  audit  function 
•Compliance  with  policies, 
regulations,  and  legislation 


Accountability 

•Clear  accountabilities  and 
responsibilities  for  due  process  and 
results 

•Delegations  consistent  with 
capabilities 


Learning,  Innovation  and  Change  Management 

•DRDC  IS  a  learning  organization  with  strategic  vision 

•Organizational  culture  anticipates  change,  values  innovation,  and  supports  transformation 
•DRDC  examines  its  own  performance  to  identify  need  for  change 
•Managers  are  empowered  through  delegation 

•Organizational  knowledge  and  memory  captured  and  managed  as  strategic  resources 


Results 

and 

Performance 


•Monitoring  and 
reporting  of 
program,  service 
and  internal 
results 

•Integrated 
financial  and  non- 
financial 
performance 
information  used 
in  decision  making 

•Reporting  is 
based  on 
measurable 
outcomes 

•Benchmark 
against  the  best 

•Transparent, 
timely  and 
accessible 
communications 
with  clients, 
partners  and 
stakeholders 


DRDC  MAF,  Indicators  Tier 
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The  Indicators  and  Measures  tiers  of  the  proposed  DRDC  MAP  were  developed  based  on 
academic  research  of  best  practices  and  theoretical  frameworks.  This  methodology  strengthens 
the  validity  of  the  individual  indicators  and  measures.  Assessment  criteria  from  Rounds  1  -  VI  of 
the  TB  MAP  were  also  utilized  in  development  of  Indicators  and  Measures,  and  efforts  were 
made  to  align  the  Indicators  and  Measures  statements  with  those  of  the  Treasury  Board  where 
appropriate. 


Governance 

and 

Strategic 

Direction 


•Confidence  of 
ADM  (S&T) 

•Coherent  policy 
agenda 

•Strong 

management 

cadre 

•Management 
framework 
supports  priority 
setting, 

reallocation,  and 
alignment  with 
government-wide 
priorities 

•Change  agenda 
addresses  all 
functional  areas 

•Organizational 
engagement  in 
administrative 
and  science- 
based  horizontal 
initiatives 


Results 

and 

Performance 


•Quality  of 
Business  Plan 
and  Annual 
Report 

•Staff  and  client 
survey  results 

•Progress  in 
strengthening 
financial  and 
program  results, 
and  performance 
measurement 

•Corporate 
monitoring  and 
review  of 
performance 

•Risk-based 
evaluation  plans 
(reviewed 
regularly)  and 
follow-up 

•Perfoimance 
against  external 
benchmarks 


DRDC  MAF,  Measures  Tier 


A  large  variety  of  corporate  data  sources  are  available  to  support  the  proposed  DRDC  MAP,  and 
integration  with  the  DRDC  business  cycle  avoids  duplicative  effort.  Additionally,  the  proposed 
DRDC  MAP  will  collect  unique  data  directly  from  DRDC  managers  by  means  of  an  on-line 
survey  instrument.  During  the  course  of  the  contract,  three  pilot  surveys  were  conducted  to  test 
the  feasibility  of  the  survey  approach  and  to  establish  baseline  assessments  of  DRDC’s 
management  cadre  against  the  core  Indicators. 

The  results  of  pilot  studies  were  generally  positive,  but  identified  areas  of  weakness.  Por 
example,  DRDC’s  performance  with  respect  to  risk  management  falls  short  of  Treasury  Board 
expectations,  in  some  measure  because  DND  has  been  slow  to  adopt  a  Corporate  Risk  Profile  as 
required  by  the  Treasury  Board.  On  the  positive  side,  questions  on  accountability  highlight  the 
importance  of  Level  2  managers  (Directors  General)  as  management  role  models,  and 
demonstrate  the  positive  effect  of  measureable  performance  standards  on  organizational 
performance. 

Significance:  The  pilot  studies  demonstrated  the  feasibility  of  management  surveys  as  a  means 
of  assessing  DRDC’s  performance  against  the  proposed  DRDC  MAP,  and  particularly  the 
performance  of  the  management  cadre.  In  this  respect,  the  proposed  DRDC  MAP  improves  upon 
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V 


the  TB  MAP  by  assessing  what  managers  actually  do,  rather  than  relying  upon  the  policies  or 
frameworks  that  establish  what  managers  should  do,  or  corporate  records  that  assess 
organizational  performance  as  a  whole. 

Recommendations:  Based  on  the  results  of  the  pilot  studies  with  the  proposed  DRDC  MAP  it  is 
recommended  that: 

•  Defence  R&D  Canada  should  adopt  the  Management  Accountability  Pramework 
proposed  in  this  report; 

•  Defence  R&D  Canada  should  take  account  of  the  findings  of  the  EXPEDITION  09 
operational  pilots  for  the  proposed  DRDC  MAP,  to  provide  a  baseline  for  the  first 
operational  cycle;  and, 

•  DRDC  should  use  future  managers’  surveys  of  the  DRDC  MAP  as  a  means  of 
determining  the  degree  of  organizational  adoption  of  the  outputs  of  the  EXPEDITION 
XX  series  of  organizational  development  projects 


VI 
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Un  cadre  de  responsabilisation  de  gestion  pour  RDDC  :  Rapport 
final 

Judy  A.  Baroni;  Warren  C.E.  Nethercote;  DRDC  Atlantic  CR  2009-136;  R  &  D 
pour  la  defense  Canada  -  Atlantique;  septembre  2009. 

Introduction  :  Recherche  et  developpement  pour  la  defense  Canada  (RDDC),  organisme  de 
service  special  du  ministere  de  la  Defense,  a  embauche  I’universite  Dalhousie  pour  1’ aider  a 
elaborer  un  cadre  de  responsabilisation  de  gestion  (CRG)  pour  EXPEDITION  09,  projet  de 
changement  organisationnel.  Ee  CRG  de  RDDC  propose,  elabore  au  titre  du  contrat,  utilise  la 
structure  du  CRG  du  CT  pour  montrer  le  rapport  entre  les  deux  cadres.  Comme  le  CRG  du  CT,  le 
CRG  de  RDDC  comporte  des  paliers  Attentes,  Indicateurs  et  Mesures,  ou  : 

•  Ees  attentes  montrent  les  normes  de  leadership; 

•  Ees  indicateurs  donnent  un  sens  pratique  aux  attentes; 

•  Ees  mesures  expriment  le  rendement. 

Comment  le  CRG  de  RDDC  differe-t-il  du  CRG  du  CT?  Sa  cible  principale  est  les  gestionnaires 
de  RDDC,  pas  le  sous-ministre. 

Void  la  «  vision  »  du  CRG  de  RDDC  propose  : 

Le  cadre  de  responsabilisation  de  gestion  (CRG)  de  RDDC  est  un  outil  de  gouvernance  qui 
facilite  le  dialogue  entre  les  gestionnaires  et  la  direction.  II  permet  de  clarifier  le  role  et  les 
responsabilites  des  gestionnaires  et  fournit  un  cadre  pour  revaluation  et  I ’amelioration  des 
processus  operationnels. 

Resultats  :  Ee  palier  Attentes  du  CRG  de  RDDC  propose  s'inspire  du  CRG  du  CT,  apres 
consultation  avec  les  directeurs  generaux  adjoints  et  les  gestionnaires  des  Services  d’entreprise. 
En  outre,  le  palier  Attentes  a  ete  legerement  modifie  durant  des  developpements  ulterieurs.  II 
reflete  les  termes  communement  utilises  et  les  circonstances  de  RDDC.  Ainsi  le  «  service  centre 
sur  le  citoyen  »  du  Conseil  du  Tresor  est-il  devenu  simple  «  service  »  dans  le  CRG  de  RDDC,  vu 
que  ni  le  MDN  ni  RDDC  ne  visent  des  services  directs  aux  citoyens. 

Ees  quatre  elements  « piliers »  constituant  le  perimetre  du  CRG  de  RDDC  etablissent 
Tenvironnement  de  RDDC,  ainsi  que  les  attitudes  et  comportements  qui  representent  le  mieux 
Torganisme.  Ees  six  elements  «  des  »  au  centre  du  CRG  s’appliquent  partout  dans  Torganisme, 
mais  plus  particulierement  encore  aux  cadres  intermediaires.  Ces  elements  des  fixent  la  norme 
pour  le  rendement  de  la  gestion.  Ees  elements  piliers  etablissent  aussi  des  normes  pour  le 
rendement  de  la  gestion,  mais  au  niveau  strategique  plutot  qu'operationnel. 
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VII 


Gouvernance 
et  orientation 
atrategique 

La  charle  de 
RDDC.Ia 
strategic  de 
science  et  de 
technologic  de 
ia  Defense  et  le 
proc^d^  de 
planification 
d’activites  du 
MDN  assurent 
un  soutien 
efficace  et  une 
autorit^ 
fonctionnelle 
dans  le 
domaine  de  la 
science  el  des 
technologies, 
pourle 

gouvernement, 
les  Forces 
canadiennes, 
la  Defense  et 
les  partenaires 
pour  la  securite 


Valeurs  de  Recherche  et  d^eloppement  pour  la  defense  Canada 

Parses  actions,  la  gestion  de  RDDC  renforce  continuellement  I'imporlance  des  valeurs  de 
I'agence  et  de  I'ethique  de  la  defense,  en  assurant  des  risultats  aux  Forces  canadiennes,  a  la 
Defense  etaux  partenaires  pour  la  securite 


Politique  et  piogiomnies 

RODC  developpe  et  mainlient 
une  capacite  de  recherche  el 
d'analysequi  luipeimelde 
fournirauMDN,auxFC,  a 
Securite  publique  Canada  et 
aux  partenaires  pour  la  securite 
des  options  de  politique,  une 
conceptionde  programme  et 
des  conseilsde  grande  qua kte. 


Employes 

RODC  a  les  employes, 
I'environnement  de  travail,  le 
leadership  etia  capacite 
vouluspourfoumirce  qui  se 
fait  de  mieux  en  science  el 
entechnologiepourla 
defense  etia  securite, 
maintenantetal'avenir. 


Lesactivitesde  S.  etT.  sent 
ax^essurle  client  et  les 
partenaires.  Les 
programmes  sontcorfus 
pourrepondrealeurs 
besoins.  Et  RODC 
encourage  etgeredefafon 
efficace  lespartenariats. 


Gestion  (les  risqiies 

RDDC  se  prepare  i  faire 
face  aux  nsquesdansun 
environnemenl  de  science 
etdetechnologle,lesgere 
ellesexplolte. 


Le  regime  de  gestion  de 
RDDC(actifs,  argent, 
employes,  services,  etc.)  est 
integre  et  efficace.  Ses 
principesde  base  sontclairs 
pourtoutle  personnel. 


Responsabilisution 

La  responsabilite  pour  les 
rdsultats,clairement 
attrlbuee,  s'aligne  avec  les 
ressourcesLesdelegatlons 
correspondenlaux 
capacites 


Gestion  de  I’apprentissage,  de  I’innovation  et  des  changements 

RDDCfavorise  la  creativite,  I'innovation  et  le  changement,  fait  la  promotion  de  I'apprentissage 
organisationnel,  prise  le  savoir  collectif  ettire  des  lemons  de  sa  performance 


Raultats  et 
performance 


RDDC  recueille 
les 

renseignement 
s  pertinents  sur 
les  resultats 
(internes, 
service  et 
programme)  et 
les  utilise  pour 
prendre  des 
decisions  au 
sein  de  RDDC. 
L'information 
diffusee  au 
ministere  et  au 
public  est 
equilibree, 
transparente  et 
facile  a 
comprendre 
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Gouvernance 
et  orientation 
strategique 


•Soutien  auSM 
MDN  parleSMA 
(S&T) 

•Cadre  de  gestion 
aligne  avecles 
resultats 
strat^giques 

•L'equipe  de 
direction  qu'tl  faut 

•Priorit^sdu 
ministere  axees 
sur  les  resultats 

•Affectation  el 
reaffectation  de 
ressourcesselon 
la  performance 

•Programme  de 
gestion  integre 

•Collaboration 

horizontale 

•Analyse  de 
I'environnement 


Valeurs  de  Recherche  et  developpement  pour  la  defense  Canada 

•Conduite  exemplaire 

•Valeursde  I’organisme  refl^t^espar la  r^alit^  etia  culture  de  ses  centres 
■Pratiques  de  gestion  axees  sur  des  valeurs 


Politique  et  progr amines 

‘Cap9«iie  d'3ri3ly$«  «t 

culture d«  ccrrsuRatian.d'exafnen  et 
de  remise  en  question 
•Politique  axee  sur  les  resultats  «t 
prioriCes  aiignees  ave«  les  priorites 
horitont^es  du  gouvememem 
•&«dua(iori  sooiale  des 
lepercussioins 

•Conlianccdu  SMA($aT).  des 
clients  et  des  partenaires 


Employes 

^ylain  d' oeuvre  comfitant  le  nonvbre  de 

personnes  ec  les  caf>dcites  voulues.tou 

en  reliant  ia  composition  de  la 

population  canadienne 

•Le  Meu  de  travail  esl  un  environnemerd 

encouiageant  et  respcctucux.ou  sont 

respect  ees  les  exigencesen  matiene  de 

laivgues  oticieRes 

•Tous  les  niveaux  de  gestion  font 

pneuve  dun  leadership  etfectif 

•Les  empfoyes  s'interessefit  a  leur 

travail,  ou  Rs  ont  Tocoasion  d'appreridre 

et  de  se  d4velopper 


•Quallle  de  service  suivie  et 
constamment  amelioree 
•Conseils  de  pointe  sur  I'exploitation 
de  la  technologie 

■Implicaiion  proactive  des  cNents  et 
partenaires 

•Rapports  effectifs  avec  cRents  et 
partenaires 


Gestion  des  lisqnes 

•Risques  geres  dans  le  contexte  du 
pnoA  de  risque  du  MON 
•Gestion  des  risques  int^ree  aux 
structures  de  plariification  et 
d'expkiitatjon  a  tous  les  niveaux  de 
Torganisme 

-Bristence  d'une  capacite  appuyam 
une  culture  de  gestion  des  risques 


•Systimes  de  gestion  foumissam  de 
I’informxioo  appropriee  et  une 
retroaction  efdcaoe  sur  les  ressources. 
les  risukais  et  les  mesures  de 
core  role 

•Gestion  de  piojet  effective,  eficace  et 
economkfue 

•Fonction  de  verificatiori  interne  etablie 
•Respect  des  politiques ,  reglements  et 
lois 


Gestion  de  I’apprentiBage,  de  I'innovation  et  des  changements 

•RDDC  estun  organisms  d'^udes  dote  d'une  vision  strategique 

•Le  culture  de  I'organisme  devance  le  changement,  appr^eltnnovation  et  soUient  la  transformation 
•RDDC  examine  son  propre  rendement  aHn  d'identifier  tout  besoin  de  changement 
•Des  dSegations  responsablhsent  les  gestionnaires 

•Le  savoir  et  la  m^oire  de  I'organisme  sont  engrangeset  geres  comme  ressources  strertegiques 


Res|)oiis<il)[liStitioii 


•Reddition  de  compte  et 
responsabilites  claires  pour 
procedure  equitable  et  resultais 
■Oelegaiion  de  pouvoir  conforme  aux 
capacites 

•Objectifs  des  subordonnes  relletant 
ceux  Rxes  a  leurs  superieurs 


Raultats  et 
performance 


•Suiviet  rapports 
pouries 

programmes,  les 
serviceset  les 
resultatsinternes 

•Renseignements 
surle  rendement 
financieretnon 
financierintegres 
etutiiisesdansla 
prise  de  decision 

■Rapports 

appuyespardes 

resultats 

mesurables 

■Criteresde 
comparaisonavec 
cequi  se  faitde 
mieux 

•Communications 

transparentes, 

opportuneset 

accessiblesavec 

clients,  partenaires 

et  parties 

prenantes 
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Les  paliers  Indicateurs  et  Mesures  du  CRG  de  RDDC  propose  ont  ete  elabores  a  partir  de 
plusieurs  elements  :  la  recherche  universitaire  sur  les  pratiques  exemplaires  et  les  cadres 
theoriques.  (methodologie  qui  renforce  la  validite  des  indicateurs  et  mesures  individuels);  et  les 
criteres  d’ evaluation  des  rondes  I  a  VI  du  CRG  du  CT  (des  efforts  etant  faits,  chaque  fois 
qu’approprie,  pour  aligner  les  enonces  d’indicateurs  et  de  mesures  avec  ceux  du  Conseil  du 
Tresor). 


Gouveinance 
et  orientation 
strategique 


Vaieurs  de  Recherche  et  developpement  pour  la  defense  Canada 

•Adhesion  du  personnel  auxvaleursde  RDDC 

•Prise  de  decision  ethlque  appuyee  par  le  Programme  d'ethique  de  la  Defense 
•Performance  de  I'organisme  dvaluee  selon  le  Programme  d'ethique  de  la  Defense 


Resultats  et 
performance 


•Conflancedu 
SMA  (S&T) 
•Prlorites 
coherentesen 
matiere  de 
politiques 
•Cadre  degesllon 
sollde 

•Le cadre de 

gestlonappule 

I'etabllssementde 

priorlles,  la 

reaffeclation  et 

I'alignementavec 

lesprioritesde 

I'ensembledu 

gouvernemeni 

•Les  prlorites  en 

matiere  de 

cbangement 

concernenttous 

lesdomalnes 

fonctionnels 

•Partcipationde 

I'organisme  aux 

Initiatives 

horizontalestant 

administratives 

quesclentiflques 


Politique  et  |)ioyr<inunes 

des  pcRiques  «t  4es 
programmes 

■nan  d«  9estion  des  ressources 
humaines  QlohsI  pofjr  le  secteurdes 
politiques 

•Inirestissement  dans  la  capacrte  en 
politk|u«s  et  dans  les  outils  de 
recherche 


•Plan  strategique  des  ressources 
humaiines  eiabt 
•Pnogres  au  cu  des  cibles  de 
ressources humaines.donit  la  qualite 
de  la  vie  au  travail ,  les  langues 
offcielles  ei  requite  pour  ks  employes 
•Progres  dans  la  mesure  de 
I'implication  des  employes  et  sort 
amelioration. 

•Quake  du  leadersh^ 

•Qualite  des  relations  de  travail 


•L'amekoration  du  service  et  la 
pianilkation  de  la  transformation  sont 
en  cours 

■La  satis  Motion  des  clients  et 
partenaires  se  mesure  annueHement 
au  vu  de  crteres 

•La  collaboration  est  essentielk  a  la 
prestalion  des  services  de  DRODC 
■RDDC  rend finfermation  technique  ei 
sclentiHque  rapidement  disponible 
pour  les  patties  prenarrtes 


Gestiondesiisqiies 

•Preuve  cjue  la  gesion  des  risques  est 

inte^ree  a  la  prise  de  dedsioa  a  la 

planifcation  et  aux  ranraits 

•Outils  de  Qestion  des  ri^ues,  fcrreibn 

et  soutien  a  cesoulilsdisponbkspourle 

personnel 

■  la  ^stion  ^it  pieuxe  de  leadership  en 
matiere  de  gesticn  desrisq^r^ 
•Capproche  de  gestbn  des  risques  est 
com^^merriaire  au  proMde  risqje  du 
ministire  dela  Defense  r^orale 
•Participation  des  pantes^rerravtes 
extemes  a  rexakaton  eta  la 
communkaiondes  risques 


•Rapports  et  dossiers  sont  opportuns  et 
exacts 

•Le  depenses  ne  depassent  pas  le 
budget 

•On  $e  procure  des  ressources  et  on 
les  utilise  judicieusement  pour  atteindre 
des  objectik 

•Les  biens  sont  inventories  et  genes  de 
tacon  strategique 

•Llnformation  s'utilise  et  se  protege  de 
on  adequAe 

•Les  realisations  attendues  sont  en 
temps  opportun  et  effectives 


Resiiorisahiliscitioii 
■Les  objectifs  de  gestion  son! 
identifiables  et  r^surables 
•les  subordonrws  ont  une  idee  claire 
des  devoirs  et  des  taches 
■Les  gestioiwaires  et  le  pcrsonr>el  ont 
la  formation  et  fautorite  approves 
pour  assumer  les  pouvoirs  qui  kurs 
sont  delegues 

•Un  processus  d'exarrven  officiel  suit  le 
progres ,  ks  delegations  et  le 
rendement 

■L'aiteinte  des  objectifs  par  ks 
subofdonnes  af^uie  ks  c^ijecttfs  de  la 
gestiort  et  la  visiw  de  rorganisine 


Gestion  de  I’apprentissage,  de  I'innowation  et  des  changements 

•progres etinvestissement  dansl'apprentlssage  da  I'organisme 
•Progresdanslespratiquesde  gestion desconnaissances 

•Perception  que  se  font  parlies  prenanteset  personnel  de  I'adaptabllite.  du  cbangement  etde 
I'innovation  au  sein  de  I'organisme 

•Liens  enire  ie  rendement  de  I'organisme  el  les  initiatives  de  developpement  de  I'organisme 


•Qualite  du  plan 

d'activitesetdu 

rapportannuel 

•Resultats  des 

sondagesaupres 

du  personnel  etde 

la  clientele 

•Progresdansle 

renforcementdes 

resultats  financiers 

etde  programme, 

ainsiquedansla 

mesure  du 

rendement 

•Sulvide 

I'organisme  et 

examendu 

rendement 

•Plans 

d'evaluallonaxes 
sur  les  risques 
(revises 

regullerement)  et 
existence  d'un 
sulvl 

•Rendementauvu 
de  criteres 
extemes 
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II  existe  toute  une  gamme  de  sources  de  donnees  d’entreprise  disponibles  pour  appuyer  le  CRG 
de  RDDC  propose;  son  integration  avec  le  cycle  d’exploitation  de  RDDC  evite  des  doubles 
emplois.  Qui  plus  est,  le  CRG  de  RDDC  propose  recueillera  des  donnees  uniques,  grace  a  un 
instrument  de  sondage  en  ligne  aupres  des  gestionnaires  de  RDDC.  Pendant  la  duree  du  contrat, 
trois  sondages  pilotes  ont  ete  effectues  pour  tester  la  faisabilite  de  I’approche  par  sondage  et  pour 
etablir  une  base  de  reference  pour  les  evaluations  des  gestionnaires  de  RDDC,  au  vu  des 
indicateurs  cles. 

Les  resultats  des  etudes  pilotes  ont  generalement  ete  positifs,  quelques  points  faibles  ayant 
toutefois  ete  identifies.  Par  exemple,  le  rendement  de  RDDC  en  matiere  de  gestion  des  risques  ne 
satisfait  pas  les  attentes  du  Conseil  du  Tresor,  en  partie  parce  que  le  MDN  a  ete  lent  a  adopter  le 
profil  de  risque  de  Porganisation  requis  par  le  Conseil  du  Tresor.  Cote  positif,  les  questions  de 
reddition  de  compte  soulignent  Timportance  des  gestionnaires  de  deuxieme  niveau  (directeur 
general)  comme  modeles  de  comportement  pour  la  gestion  et  montrent  les  effets  positifs  de 
normes  de  rendement  mesurables  sur  le  rendement  de  I’organisme 
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Importance  :  Les  etudes  pilotes  out  montre  la  faisabilite  de  sondages  aupres  de  la  gestion  comme 
moyen  d’ evaluation  du  rendement  de  RDDC  au  vu  du  CRG  de  RDDC  propose,  notamment  pour 
le  rendement  des  gestionnaires.  Sur  ce  plan,  le  CRG  de  RDDC  propose  ameliore  le  CRG  du  CT, 
en  evaluant  ce  que  font  effectivement  les  gestionnaires,  plutot  que  d’evaluer  soit  les  politiques  ou 
cadres  etablissant  ce  qu’ils  sont  censes  faire,  soit  les  dossiers  de  Torganisme  et  le  rendement  de 
I'organisme  dans  son  ensemble. 

Recommandations  :  Au  vu  des  resultats  des  etudes  pilotes  sur  le  CRG  de  RDDC  propose,  voici 
nos  recommandations  : 

•  Recherche  et  developpement  pour  la  defense  Canada  devrait  adopter  le  cadre  de 
responsabilisation  de  la  gestion  propose  dans  le  present  rapport; 

•  Recherche  et  developpement  pour  la  defense  Canada  devrait  tenir  compte  des 
constatations  des  projets  pilotes  operationnels  d’EXPEDITlON  09  pour  le  CRG  de 
RDDC  propose,  afm  d’ avoir  une  base  de  reference  pour  le  premier  cycle  operationnel; 

•  RDDC  devrait  utiliser  les  futurs  sondages  aupres  des  gestionnaires  du  CRG  de  RDDC 
pour  determiner  jusqu’a  quel  point  I’organisme  adopte  les  extrants  de  la  serie  de  projets 
de  developpement  organisationnel  EXPEDITION  XX. 
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1  Introduction 


Defence  Research  and  Development  Canada  (DRDC)’,  a  special  operating  agency  of  the 
Department  of  National  Defence,  contracted  Dalhousie  University  to  support  the  development  of 
a  management  accountability  framework  (MAP)  as  part  of  DRDC’s  EXPEDITION  09  project. 
This  is  the  final  report  of  the  contract  and  provides  the  recommended  approach  to  a  DRDC  MAE. 
Further  details  of  the  MAE  development  are  available  in  related  documents.^ 

1.1  EXPEDITION  09 

EXPEDITION  09  is  DRDC’s  core  organizational  change  initiative,  following  an  earlier  change 
initiative,  EXPEDITION  07.  The  EXPEDITION  series  established  the  principle  of  an  ‘umbrella,’ 
agency-wide  organizational  development  program,  under  the  control  of  the  DRDC  Executive 
Committee,  demonstrating  organizational  support.  Figure  1  illustrates  the  project  structure  of 
EXPEDITION  09  schematically.  This  contract  report  supports  Task  3  ofWBE  1.4:  Management 
Eeadership  Capabilities. 


WBE  i 


EXPEDITION  09 


WBE  1.4: 

Management  Leadership  Capabilities 


Task  1:  Management  Workload 


Task  2:  Management  Leadership 
Development 


Task  3:  Management  Accountability 
Framework 


ZL 

WBE  n 


Figure  1:  Expedition  09  Project  Structure 


’  A  list  of  acronyms  and  initialisms  is  included  on  page  71  of  this  report. 

^  See  Nethercote,  W.C.E.,  “A  DRDC  Management  Accountability  Framework  -  First  Thoughts,”  Dalhousie  University 
School  of  Public  Administration,  Discussion  paper,  28  February  2008;  Nethercote,  W.C.E.,  “A  DRDC  Management 
Accountability  Framework:  A  Straw-man,  April  2008,”  Dalhousie  University  School  of  Public  Administration, 
Discussion  Paper,  April  21,  2008;  O’Blenis,  Craig  and  Nethercote,  W.C.E.  “A  DRDC  Management  Accountability 
Framework,  Cycle  1  Final  Report,”  Dalhousie  University  School  of  Public  Administration,  DRDC  Atlantic  Contract 
Report,  CR  2008-188,  October  2008;  Baroni,  Judy  A.  and  Nethercote,  W.C.E. ,  “A  DRDC  Management  Accountability 
Framework:  Results  of  Cycle  2,”  Dalhousie  University  School  of  Public  Administration,  DRDC  Atlantic  Contract 
Report,  CR2009-135,  November  2009. 
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1 .2  Scope  of  this  Report 

This  report  provides  an  overview  of  the  proposed  DRDC  MAP,  concentrating  on  its  features  and 
its  application,  rather  than  its  development.  The  report  includes  reference  to  the  Treasury  Board 
MAP,  to  provide  context  for  the  DRDC  MAP. 

Details  of  the  development  of  the  MAP  are  provided  in  reports  by  Nethercote,  O’Blenis  and 
Nethercote,  and  Baroni  and  Nethercote.^ 

1 .3  Structure  of  this  Report 

Section  2  provides  background  to  the  development  of  the  DRDC  MAP,  beginning  with  a 
description  of  the  Treasury  Board  (TB)  MAP,  followed  by  discussion  of  issues  with  it  and  then, 
by  the  principles  of  a  DRDC  MAP.  Section  3  includes  reference  to  the  literature,  but  more 
complete  discussion  is  available  in  companion  reports."* 

Section  3  opens  with  a  vision  of  a  steady-state  DRDC  MAP  and  then  describes  the  structure  of 
the  DRDC  MAP  in  two  parts,  the  pillar  and  core  elements;  see  Pigure  2.  The  vision  provides  a 
strategic  overview  of  the  characteristics  of  a  DRDC  MAP  of  interest  to  DRDC’s  executive  cadre: 
the  purpose  of  the  MAP;  the  business  processes  of  the  MAP;  the  participants  in  MAP  processes; 
and  the  resources  required  by  the  MAP  processes.  The  pillar  elements  of  the  DRDC  MAP  set  the 
environment  of  DRDC  and  the  attitudes  and  behaviours  that  best  represent  the  organization.  The 
core  elements  apply  across  the  organization,  but  are  especially  relevant  to  middle  managers.  The 
core  elements  set  the  standards  for  the  performance  of  management.  The  pillar  elements  also  set 
standards  for  the  performance  of  management,  but  more  at  the  strategic  than  the  operational  level. 


Pillar  Elements: 

DRDC  Agency  Values 
Governance  &  Strategic 
Direction 

Learning,  Innovation  & 
Change 
Management 
Results  &  Performance 


Core  Elements: 

Policy  &  Programs 

People 

Service 

Risk  Management 

Stewardship 

Accountability 


Figure  2:  Pillar  and  Core  Elements  of  the  DRDC  MAF,  in  the  TB  Style 

Section  4  discusses  the  application  of  the  DRDC  MAP,  beginning  with  the  results  of  the  pilot 
studies  undertaken  during  EXPEDITION  09.  These  pilot  studies,  although  limited  in  scope. 


^  See  O’Blenis,  Craig  and  Nethercote,  W.C.E.  “A  DRDC  Management  Accountability  Framework,  Cycle  1  Final 
Report,”  Dalhousie  University  School  of  Public  Administration,  DRDC  Atlantic  Contract  Report,  CR  2008-188, 
October  2008,  and  Baroni,  Judy  A.  and  Nethercote,  W.C.E. ,  “A  DRDC  Management  Accountability  Framework: 
Results  of  Cycle  2,”  Dalhousie  University  School  of  Public  Administration,  DRDC  Atlantic  Contract  Report,  CR2009- 
135,  November  2009. 
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provide  indication  of  both  strengths  and  weaknesses  in  management  practices  in  DRDC.  Section 
4  closes  with  proposed  approaches  for  operational  application  of  a  DRDC  MAP  involving  both 
the  use  of  surveys  of  managers  and  the  exploitation  of  corporately-held  information. 

The  main  body  of  the  report  closes  with  Section  5  which  provides  recommendations  and 
conclusions. 

A  discussion  of  risk  management  practices  in  DRDC  in  the  context  of  organizational  culture 
theory  is  included  as  an  Annex^  to  the  report,  as  a  first  step  in  addressing  the  conundrum  of  public 
sector  S&T  organizations.  How  can  the  public  sector  penchant  for  risk  reduction,  if  not  even  risk 
avoidance,  be  reconciled  with  a  science  and  technology  (S&T)  organization’s  need  for  risk 
exploitation?  The  Annex  concludes  with  recommended  bases  for  a  training  plan  that  would 
enhance  risk  management  in  DRDC. 

A  bibliography  lists  sources  cited  in  the  main  body  of  this  report;  sources  for  Annex  A  are  listed 
separately. 


^  Kevin  Quigley,  “Using  Cultural  Theory  to  Examine  Risk  Management  Practices:  A  Commentary  for  Defence 
Research  and  Development  Canada,”  see  p.  51  of  this  report. 
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2  Background^ 


Since  1997,  the  Treasury  Board  and  its  Secretariat  have  been  the  designated  management  board 
for  the  Government  of  Canada,  with  a  variety  of  roles  including  “leading  and  providing  expertise 
in  the  development  of  an  agenda  to  improve  management  practices.”^  This  management 
improvement  agenda  expressed  in  Results  for  Canadians  is  arguably  the  genesis  of  the  TB 
Management  Accountability  Framework. 

One  element  of  Results  for  Canadians  was  the  Modem  Comptrollership  Initiative  (MCI),  which 
called  for  “more  effective  decision-making,  greater  accountability,  a  mature  approach  to  risk 
management,  results-based  control  systems,  and  shared  values  and  ethics.”* *  Strong  and  clear 
accountability  was  a  central  element  of  the  MCI,  to  ensure  that  the  four  management 
commitments  of  the  Government  of  Canada  would  be  met:  Citizen  Focus;  Values;  Results;  and 
Responsible  Spending. 

The  Management  Accountability  Framework  was  the  successor  initiative  to  the  MCI,  subsuming 
modem  comptrollership  along  with  other  key  elements  of  modem  management  to  provide 
management  expectations  for  deputy  heads  and  public  service  managers.^ 


2.1  The  TB  MAF:  Rationale  and  History 


In  2003,  the  Treasury  Board  introduced  a  draft  MAF  with  10  core  elements  of  public  service 
management  for  bilateral  discussions  between  deputy  ministers  and  the  Secretary  of  the  Treasury 
Board.'®  The  MAF  uses  an  integrative  approach,  combining  10  elements  to  show  what  the 
Treasury  Board,  and  the  public  indirectly,  expects  of  managers.  These  10  elements  are  meant  to 
succinctly  capture  what  Jim  Judd,  Secretary  of  the  Treasury  Board  Secretariat  at  the  time,  called 
“the  questions  that  keep  Deputy  Ministers  up  at  night”.  " 

•  Governance  and  Strategic  Directions  •  Public  Service  Values 

•  Policy  and  Programs  •  People 


•  Risk  Management 


•  Stewardship 


•  Results  and  Performance 

•  Citizen-focused  Service 


•  Learning,  Innovation  and  Change 
Management 

•  Accountability 


*  Editor’s  note:  unless  otherwise  noted,  the  internet  links  in  this  report  were  current  as  of  30  June  2010. 

’  Treasury  Board  of  Canada  Secretariat.  Results  for  Canadians:  A  Management  Framework  for  the  Government  of 
Canada,  March  30,  2000,  p.  18. 

*  Depuis,  Jean,  Modern  Comptrollership  and  the  Management  Accountability  Framework,  Parliamentary  Information 
and  Research  Service,  Library  of  Parliament  PRB  06-23E,  Ottawa,  April  3,  2006,  p.  3. 

*  Depuis,  p.  9. 

Treasury  Board  of  Canada  Secretariat,  “MAF  Assessments  -  Round  I,”  (Treasury  Board  of  Canada  Secretariat, 
2007),  http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/assessments-evaluations  e.asp#rl. 

’*  Ivan  Blake,  Management  Accountability  Framework,  (Treasury  Board  of  Canada,  Secretariat:  2005),  http://www.tbs- 
sct.gc.ca/maf-crg/documents/video- video/video-video  e.asp. 
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In  2004,  the  Treasury  Board,  with  35  federal  departments  and  agencies,  developed  management 
priorities  based  on  the  ten  elements.’^  From  this,  the  Treasury  Board  identified  41  indicators  and 
in  2005,  it  assessed  53  departments  and  agencies  against  the  indicators.  At  this  time,  the  Treasury 
Board  did  not  intend  to  review  management  quality;  instead,  it  was  looking  for  performance 
compared  to  the  specific  indicators.'^  The  latest  round  of  assessments,  conducted  in  2006, 
expanded  to  55  departments  and  agencies.'"'  The  Treasury  Board  revised  the  2004  indicators 
based  on  departmental  and  agency  input, and  to  focus  on  management  aspects  it  could  assess 
easily  through  normal  oversight  activities.  Information  sources  included  estimates,  management 
resources  and  results  structures  (e.g.,  program  activity  architectures.  Memoranda  to  Cabinet,  and 
Treasury  Board  submissions),  and  existing  consultations.'® 


Figure  3:  TB  MAF,  Expectations  Tier 


Treasury  Board  of  Canada  Secretariat,  “MAF  Assessments  -  Round  II,”  (Treasury  Board  of  Canada  Secretariat, 
2007),  http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/assessments-evaluations  e.asp#r2. 

Treasury  Board  of  Canada  Secretariat,  “Round  III  (2005),”  (Treasury  Board  of  Canada  Secretariat,  2007), 
http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/2005/index  e.asp. 

Treasury  Board  of  Canada  Secretariat,  “List  of  Departments  and  Agencies  assessed  during  MAF  Round  IV,” 
(Treasury  Board  of  Canada  Secretariat,  2007),  http://www.tbs-sct.gc.ca/maf-crg/assessments- 
evaluations/20Q6/departments-ministeres  e.asp. 

Treasury  Board  of  Canada  Secretariat,  “MAF  Assessments  -  Round  IV,”  (Treasury  Board  of  Canada  Secretariat, 
2007),  http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/assessments-evaluations  e.asp#r4. 

Treasury  Board  of  Canada  Secretariat,  “MAF  Elements  and  Indicators  -  2006,”  (Treasury  Board  of  Canada 
Secretariat,  2007),  http://www.tbs-sct.gc.ca/maf-crg/indicators-indicateurs/2006/elements-elements  e.asp. 
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The  TB  MAP  has  three  tiers  which  detail  the  commitments  senior  managers  make  as 
departmental  leaders.  The  top  tier,  Expectations,  defines  leadership  standards  and 
recognizes  that  successful  management  relies  on  the  connection  between  each  element.  The 
second  tier.  Indicators,  supports  Expectations,  providing  practical  descriptions  easily 
adapted  to  an  agency’s  unique  context.  The  bottom  tier.  Measurements,  is  an  expression  of 
each  element’s  performance  standards.  While  the  expectations  are  constant,  a  MAE 
recognizes  departments  and  agencies  are  unique  by  allowing  flexibility  in  definition  of 
Indicators  and  Measures.’’ 

As  Figure  3  shows,  governance  and  strategic  directions,  public  service  values,  learning, 
innovation  and  change  management,  and  results  and  performance  frame  the  MAE.  These 
elements  guide  good  management,  enclosing  the  elements  required  to  make  good  decisions.’*  In 
essence,  the  elements  focus  on  the  capacity  and  capability  within  a  department.  The  MAE  does 
not  address  a  department’s  relationship  with  its  Minister,  nor  does  it  discuss  horizontal 
collaboration.’®  These  limitations  result  from  the  scope  of  the  MAE,  which  provides  for  a 
dialogue  between  the  Treasury  Board  and  senior  managers  on  the  expectations  of  departmental 
leadership 

Expectations  Tier 

The  elements  of  the  Expectations  tier  in  Figure  3  describe  the  Treasury  Board’s  expectations  of 
senior  managers  in  the  Government  of  Canada.  This  framework  overcomes  silo  thinking  by 
emphasizing  the  relationship  between  each  of  the  elements.  For  example,  although  Stewardship 
can  be  narrowly  considered  a  financial  activity,  managing  human  resources  has  significant  impact 
on  financial  performance.  In  addition  to  the  People  element.  Policy  and  Program  directives 
connect  intimately  to  Stewardship  decisions.  A  manager  may  find  that  project  failure  is  due  to 
limited  resources,  but  without  sound  financial  management,  it  is  a  challenge  to  overcome  the 
problem.  A  manager  succeeds  at  delivering  high-quality  public  service  by  meeting  the 
expectations  of  each  element.  To  meet  expectations,  managers  need  performance  indicators  and 
clear  measures,  detailed  in  the  remaining  MAE  tiers. 

Indicators  Tier 

The  indicators  describe  the  breadth  of  the  elements.  The  Treasury  Board  intends  that  Deputy 
Ministers  engage  their  executive  councils  in  discussions  of  the  items  listed  in  the  indicators  to 
determine  how  well  their  department  reflects  the  MAF’s  expectations. 

Measures  Tier 


Measures  are  used  to  determine  how  a  department  functions;  unlike  Expectations  and  Indicators, 
Measures  may  change  in  response  to  departmental  needs.  Sources  of  information  change, 
particularly  when  reporting  methods  change.  The  Measures  tier  of  the  Treasury  Board  MAE  lists 
potential  sources,  while  acknowledging  a  department  may  have  other,  accurate  information 


Recently,  Rounds  V  and  VI  of  the  TB  MAP  referred  to  the  Indicators  and  measures  as  ‘Areas  of  Management’  and 
‘Lines  of  Evidence,’  respectively.  The  present  report  will  continue  to  use  Indicators  and  Measures  since  the  TBS 
website  continues  to  use  these  terms  in  its  descriptive  documentation  of  the  MAP.  Regardless,  ‘Areas  of  Management’ 
and  ‘Lines  of  Evidence’  are  arguably  more  descriptive. 

Ivan  Blake,  Management  Accountability  Framework,  (Treasury  Board  of  Canada,  Secretariat:  2005),  http://www.tbs- 
sct.gc.ca/maf-crg/documents/video-video/video-video  e.asp. 

Ibid. 
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sources  that  describe  departmental  performance.  The  Treasury  Board  and  Privy  Council  Office 
use  information  found  through  the  Measures  to  engage  Deputy  Ministers  on  their  department’s 
performance.  Depending  on  performance,  certain  measures  may  be  more  important  for 
discussion  than  others.  For  example,  a  department  may  perform  well  at  risk  management  but 
could  benefit  from  improvement  in  its  accountability  guidelines.  In  this  case,  the  Treasury  Board, 
the  Privy  Council  Office,  and  the  Deputy  Minister  would  focus  on  accountability  at  the  expense 
of  risk  management. 

2.2  Issues  with  the  TB  MAF 

While  the  MAF  offers  a  useful  overview  of  the  Treasury  Board’s  expectations  for  senior 
management,  it  cannot  capture  the  details  of  decision  making  environments  and  pressures  which 
are  unique  to  each  institution,  nor  does  it  offer  a  methodology  for  assessing  the  quality  of 
programming.  In  this  sense  the  TB  MAF  is  neither  a  replacement  for  assessing  DM  performance 
through  the  Committee  of  Senior  officials  (COSO),  nor  an  alternative  to  program  evaluations, 
although  it  does  offer  valuable  perspectives  from  which  to  view  each  of  these  areas. 

As  a  guide  to  good  management  practices,  the  elements  focus  on  organizational  capacity  and 
capability  within  a  department^';  however,  the  MAF  does  not  address  a  department’s  relationship 
with  its  Minister  and  it  does  not  discuss  horizontal  collaboration.^^  These  limitations  result  from 
the  scope  of  the  MAF,  which  provides  for  a  dialogue  between  the  Treasury  Board  Secretariat  and 
Deputy  Ministers  on  the  expectations  of  departmental  leadership. 

The  MAF  was  originally  intended  as  a  ‘code  of  excellence’  type  document, demonstrating 
sound  management  practices  which  would  rely  heavily  on  the  commitment  of  senior  managers 
(DM  and  ADM)  to  be  effective;  however,  as  the  framework  has  evolved  and  its  use  adopted  by 
all  departments,  it  has  become  “an  instrument  for  management  oversight,”  relying  more  heavily 
on  the  reporting  capabilities  of  subordinates.  Theoretically,  the  benefit  of  this  alternative 
approach  is  that  by  having  better  defined  and  mandatory  reporting  measures,  there  is  increased 
and  better  comparability  between  departments;  however,  as  noted  above,  each  department  has 
unique  environments  and  mandates  which  make  direct  comparison  difficult. 

One  drawback  in  the  shift  from  ‘code  of  excellence’  to  an  oversight  tool  is  that  by  redefining 
MAF  as  a  reporting  structure,  it  is  effectively  alienated  from  being  used  as  a  strategic  tool,  and 
places  more  burden  on  an  already  reporting-heavy  system: 

“...the  cost  of  moving  too  far  in  this  direction  is  that  many  deputy  ministers 
will  create  sufficient  capability  to  credibly  feed  reporting  requirements,  but 


Lee  McCormack,  Using  Evaluation  in  the  Canadian  Expenditure  Management  System,  OECD  Presentation, 
Washington,  DC,  June  1 1-12,  2007.  Available  from:  httri://www.oecd.org/dataoecd/21/9/39793340.t)df. 

Ivan  Blake,  Management  Accountability  Framework,  (Treasury  Board  of  Canada,  Secretariat:  2005),  http://www.tbs- 
sct.gc.ca/maf-crg/documents/video-video/video-video  e.asp. 

“  Ibid. 

Evert  Lindquist,  “How  Ottawa  Assesses  Departmental/  Agency  Performance:  Treasury  Board’s  Management 
Accountability  Framework”  in  How  Ottawa  Spends,  2009  -  2010:  Economic  Upheaval  and  Political  Dysfunction, 
Allan  M.  Maslove,  ed.  Forthcoming,  October  2009. 

TBS,  MAF  Implementation,  Available  from:  http://www.tbs-sct.gc.ca/maf-crg/implementation- 
implementation/implementation-implementation-eng.asp. 
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effectively  ‘quarantine’  it  [the  MAP]  from  strategic  dialogue  and  decision¬ 
making  in  departments  focused  on  dealing  with  pressing  challenges. 

Another  issue  with  the  MAP  is  that  it  can  portray  a  higher  level  of  departmental  accountability 
than  truly  exists.  This  weakness  stems  from  the  TBS  dependence  on  each  department  to  self- 
report  by  providing  information  on  MAP  measures,  a  lack  of  external  reviews,  as  well  as  a  lack  of 
TBS  capacity  to  provide  support  to  departments  after  the  assessment  to  close  gaps  in  MAP 
performance  areas.  Departments  which  become  better  at  responding  to  the  TB  MAP  reporting 
structure  can  be  given  improved  MAP  assessments  without  actually  improving. 

The  validity  of  MAP  measures  and  the  assessment  scoring  system  are  also  problematic  issues. 
There  is  no  information  available  to  explain  the  TBS’  methodology  in  selecting  indicators  and 
measures  for  each  Element.  The  departmental  assessment  against  each  measure  is  largely 
dependent  on  qualitative  judgments,  and  is  more  concerned  with  the  existence  and 
implementation  of  policies  and  frameworks  than  with  the  impact  of  those  policies  and 
frameworks.  The  implicit  assumption  is  that  the  mere  existence  of  policies  and  procedures  within 
a  department  will  lead  to  improved  results.^’ 

Two  weaknesses  in  the  scoring  system  are  that  each  indicator  is  given  equal  weight  in  the 
assessment  of  an  element  and  each  element  is  weighted  equally  for  all  departments.^*  Weak 
performance  against  a  particular  indicator  may  not  justify  a  weakened  assessment  in  the  element 
overall,  and  furthermore,  departmental  mandates  may  dictate  that  a  particular  element  be 
weighted  more  heavily^^  (the  service  element  for  Service  Canada,  for  example).  Note  that  there 
is  here  is  no  overall  score  for  the  MAP,  only  individual  elemental  assessments;  the  reader  is  left 
to  draw  his/her  own  conclusions  as  to  the  overall  departmental  performance  against  the  MAP. 

While  the  TB  MAP  was  not  intended  to  evaluate  DM  leadership  per  se,  judgment  is  implied, 
particularly  because  successful  performance  of  the  department  in  each  area  relies  heavily  on 
senior  management  performance.  Not  only  is  it  awkward  for  subordinates  to  be,  in  effect, 
evaluating  the  DM’s  leadership  performance  when  conducting  a  MAP  assessment,  the  MAP 
measures  do  not  take  into  account  the  conflicting  priorities,  strategic  choices  and  unique  pressures 
faced  by  each  DM.*® 

Pinally,  the  costs  associated  with  MAP  are  thought  to  be  quite  high  in  relation  to  its  benefits;  one 
conservative  estimate  places  the  cost  to  TBS  at  $42  million  annually.*'  This  figure  includes 
increased  overhead  costs,  people  costs  and  opportunity  costs. 

In  conclusion,  the  TB  MAP  is  a  good  checklist  against  which  departments  can  assess  their 
performance,  and  the  indicators  and  measures  are  adequate  starting  points  for  generic 
assessments;  however,  each  department  or  agency  must  be  sensitive  to  their  particular 


^  Evert  Lindquist,  Forthcoming,  October  2009. 

Ibid. 

Melvin  J.  Dubnick  and  Jonathan  B.  Justice,  Accounting  for  Accountability,  Discussion  Paper,  2004  Annual  Meeting 
of  the  American  Political  Science  Association,  3. 

Evert  Lindquist,  Forthcoming,  October  2009. 

Ibid. 

Ibid. 

Ibid. 
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circumstances  when  self-assessing  against  the  MAP.  It  is  preferable  for  each  organization  to 
institute  their  own  MAP,  based  on  and  aligned  with  the  TB  MAP,  as  a  mechanism  for  assessing 
performance  in  order  to  report  effectively  to  superior  bodies  and  to  improve  their  own 
management  performance. 

Table  1  outlines  the  five  main  problem  areas  with  the  TB  MAP  and  potential  strategies  to 
mitigate  these  problems  with  the  DRDC  MAP. 


Table  1:  MAF  Problem  Areas  and  Possible  Mitigation  Strategies 


TB  MAF  Problem  Area 

Mitigation  Strategy  for  a  DRDC  MAF 

Loss  of  Strategic  Tool 

-Discuss  MAF  as  it  relates  to  organizational  priorities  at  RDEC^^  retreats 
-select  relevant  information  sources  and  tools 

-executive  summary  of  MAF  assessments  focuses  on  key  strategic  themes 
and  recommendations 

Reporting  Burden 

-Integrate  MAF  reporting  with  PAA^^  and  other  Corporate  functions 

Appearance  of 

Accountability 

-Select  relevant  measures  and  information  sources 
-Document  methodology  so  that  the  same  approach  is  utilized  each  year 
-Set  goals  for  the  next  assessment  at  the  end  of  the  previous  assessment 
-Use  managers’  survey  to  incorporate  information  on  actual  working 
conditions 

Measures  and  Assessment 

-  Utilize  solid  academic  theory  as  basis  for  selection  of  measures  and 

development  of  survey  questions 

-Select  relevant  information  sources  and  tools 

Assessing  leadership 

-Do  not  utilize  MAF  assessments  as  a  performance  measurement  tool 
-Ensure  recommendations  are  aimed  at  organizational  level,  not  individuals 

Costs 

-Utilize  data  mining  techniques  to  get  relevant  information  from  pre-existing 
sources 

-Refine  other  reporting  systems  to  capture  MAF  related  measures  when 
other  data  is  collected 

-Select  relevant  measures  and  information  sources  to  cut  down  on 
unnecessary  reporting 

2.3  Principles  of  a  DRDC  MAF 

A  DRDC  MAP  must  satisfy  the  following  conditions  in  order  to  be  adopted  willingly  by  the 
organization: 

•  Support  ADM(S&T)’s  contributions  to  the  TB  MAP  process; 

•  Support  dialogue  that  clarifies  roles  and  responsibilities  throughout  the  chain-of- 
command; 

•  Support  assessment  and  improvement  of  business  processes; 


RDEC:  Research  and  Development  Executive  Committee. 
PAA:  Program  Activity  Architecture 
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•  Exploit  existing  corporate  information  sources  where  possible;  and, 

•  Minimize  workload  where  direct  data  collection  from  managers  is  required. 

A  DRDC  MAE  should  also  reflect,  and  thereby  promote  DRDC’s  Mission,  Vision  and  Values,  as 
well  as  the  Defence  S&T  Strategy. 

Mission 

DRDC's  mission  is  to  ensure  that  the  Canadian  Forces  are  technologically  prepared  and 
operationally  relevant  by: 

•  Providing  expert  science  and  technology  advice  to  the  Canadian  Forces  and  the 
Department  of  National  Defence; 

•  Conducting  research,  development  and  analysis  to  contribute  to  new  and  improved 
defence  capabilities; 

•  Anticipating  and  advising  on  future  science  and  technology  trends,  threats  and 
opportunities; 

•  Engaging  industrial,  academic  and  international  partners  in  the  generation  and 
commercialization  of  technology;  and, 

•  Providing  science  and  technology  for  external  customers  to  enhance  defence  science  and 
technology  capacity. 

Visipn^^ 

DRDC’s  vision  is  to  be  known  worldwide  as  the  best  in  science  and  technology  for  defence  and 
security. 

Values^'^ 

DRDC’s  values  guide  how  we  accomplish  our  mission  and  maintain  excellence  in  science: 

Commitment'.  We  demonstrate  dedication  and  pride  in  working  towards  our  vision. 

Client  Focus:  We  bring  excellence  to  clients,  both  internal  and  external,  by  focusing  efforts 
on  discovering  and  meeting  their  needs. 

Creativity  and  Innovation:  We  generate  innovative  solutions,  approaches,  products  and 
services  that  improve  the  status  quo. 

Leadership:  We  actively  and  enthusiastically  seek  to  exert  influence  and  originate  action  to 
achieve  our  goals. 

Professionalism  and  Integrity:  We  focus  our  effort  on  achieving  quality  results  and  we 
behave  in  an  honest,  ethical  manner,  dealing  with  others  respectfully  and  fairly. 

Trust  and  Respect:  We  are  open,  honest  and  responsible  in  our  relationships  and  we 
recognize  and  value  the  contributions  of  others. 


Direct  quote  from  DRDC  Annual  Report  for  the  year  ending  3 1  March  2008:  Shaping  Defence  and  Security 
Capabilities  Through  Science  and  Technology,  p.  3. 

Ibid. 

Ibid. 
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Teamwork'.  We  demonstrate  effective  interpersonal  skills,  and  work  cooperatively  and 
productively  within  and  across  DRDC  to  achieve  common  goals. 


The  Defence  S&T  Strategy  includes  four  interdependent  action  areas,  each  associated  with 
specific  deliverables^’.  The  core  elements  of  the  DRDC  MAP  should  examine  management 
performance  related  to  these  S&T  deliverables,  as  shown  in  Table  2,  overleaf 

The  Defence  S&T  Strategy  also  describes  six  attributes  the  Organization  must  cultivate  in  order 
to  be  successful  in  achieving  maximum  Impact  in  the  identified  Action  Areas. All  six  are 
dependent  on  the  existence  of  a  solid  Governance  Structure. 

•  Linkage  -  through  the  Enterprise,  Departmental  and  Canadian  Forces  organizations  are 
effectively  positioned  and  synchronized  to  contribute  to  the  direction,  delivery  and 
exploitation  of  S&T  outputs. 

•  Innovation  -  the  Enterprise,  with  its  external  partners,  produces  innovative,  cost-effective 
solutions  or  novel  alternatives  to  the  highest  priority  defence  and  security  needs  that  are 
appropriate  to  the  Canadian  context. 

•  Leverage  -  the  Enterprise  attracts  top-tier  national  and  international  S&T  partners  so  as  to 
leverage  their  resources,  knowledge,  experience  and  technology. 

•  Agility  -  the  Enterprise  is  appropriately  agile  so  as  to  identify  and  respond  effectively  to 
new  threats  and  opportunities  that  derive  from  the  global  advancement  of  S&T  or  from 
the  rapidly  evolving  defence  and  security  environment. 

•  Balance  -  the  S&T  outputs  produced  by  the  Enterprise  are  suitably  balanced  from 
multiple  perspectives.  These  outputs  address  the  prioritized  needs  from  the  perspectives 
of  core  departmental  processes,  Canadian  Forces  and  environment-specific  capability 
requirements.  They  address  the  needs  across  multiple  time  horizons,  from  responding  to 
the  challenges  of  today's  operations  to  shaping  the  Canadian  Forces  of  the  future.  Finally, 
they  sustain  core  internal  S&T  capabilities  across  as  broad  a  spectrum  as  resources  allow 
while  ensuring  the  critical  mass  necessary  in  each  area  to  produce  world  calibre  results. 

•  Excellence  -  the  Enterprise  consistently  produces  results  of  the  highest  possible  quality 
and  credibility  built  on  niche  world-calibre  internal  S&T  capabilities  and  strong,  effective 
external  partnerships. 

The  Indicators  and  Measures  of  a  DRDC  MAF  should  assess  the  strength  of  these  six  attributes, 
either  directly  or  indirectly. 


Defence  S&T  Strategy,  2006.  pp.  20-22. 
^**Ibid.pp.  19-20. 
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Table  2:  S&T  Strategy  Areas  and  Deliverables 


Action  Area 

S&T  Deliverables 

Key  MAP  Core  Elements 

Establish  an 
integrated 
governance 
mechanism 

Defence  S&T  Enterprise  Charter 

Accountability;  Risk  Management 

Defence  S&T  communications  plan 

Service 

Process  model  for  the  S&T  Enterprise 
Annual  Business  Cycle 

Accountability 

Aligned  Research,  Technology  and 
Analysis  Program  management 

Accountability 

Develop  a  "full- 
service"  defence 

S&T  capability 

S&T  capability  assessment  of  the  S&T 
Enterprise 

Policy  and  Programs 

Human  resources  plan  for  the  S&T 
Enterprise 

People 

Infrastructure  plan  for  the  S&T 

Enterprise 

Stewardship 

Build  strategic 
partnerships 

Process  model  for  managing 
partnerships 

Service 

Partnering  arrangements  among  S&T 
Enterprise  members 

People;  Service 

International  partnering  framework  and 
supporting  agreements 

Policy  and  Programs;  People 

Partnering  framework  for  industrial, 
academic  and  other  government 
departments 

Policy  and  Programs 

Establish  Enablers 

Information  management/information 
technology  capability  to  support  the 

S&T  Enterprise 

Risk  Management;  Stewardship 

Harmonized  procurement  practices 
across  research,  development  and 
acquisition 

Stewardship 

Intellectual  property  management 
principles 

Stewardship 

Departmental  approach  to  technology 
insertion 

Risk  Management;  Stewardship 
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3  Structure  of  the  DRDC  MAF 


This  section  begins  with  a  vision  of  a  steady-state  DRDC  MAF,  to  provide  a  strategic  overview  of 
the  characteristics  of  a  DRDC  MAF  of  interest  to  DRDC’s  executive  cadre:  the  purpose  of  the 
MAF;  the  business  processes  of  the  MAF;  the  participants  in  MAF  processes;  and  the  resources 
required  by  the  MAF  processes.  An  overview  of  the  Expectations  Tier  of  the  DRDC  MAF  is  then 
presented,  followed  by  discussion  of  the  pillar  and  core  elements  of  the  DRDC  MAF.  This 
discussion  of  elements  includes  a  description  of  elements  for  the  Indicators  and  Measures  Tiers 
and  of  proposed  instruments  to  allow  data  collection  for  the  measures. 

The  proposed  DRDC  MAF  uses  the  structure  of  the  TB  MAF  to  demonstrate  the  relationship 
between  the  two  frameworks.  Like  the  TB  MAF,  the  proposed  DRDC  MAF  has  Expectations, 
Indicators,  and  Measures  tiers,  where: 

•  Expectations  demonstrate  leadership  standards, 

•  Indicators  convey  practical  meaning  of  the  Expectations,  and 

•  Measures  are  expressions  of  performance. 

Each  tier  of  the  proposed  DRDC  MAF  has  ten  elements  that  describe  key  characteristics  of 
management  performance,  in  which  four  pillar  elements  surround  six  core  elements.  This  layout 
helps  describe  the  relationship  of  the  elements.  As  shown  in  Figure  4,  overleaf,  the  MAF  begins 
on  the  left  hand  side,  with  Governance  and  Strategic  Direction  and  ends  on  the  right  side  with 
Results  and  Performance.  The  directional  arrows  which  take  the  observer  from  the  left  to  the 
right  of  the  MAF  cross  through  the  six  core  elements,  which  are  flanked  by  the  two  remaining 
pillar  elements.  This  layout  suggests  that  the  six  core  elements,  bounded  by  Defence  R&D 
Canada  Agency  Values  and  Learning,  Innovation  and  Change  Management  principles,  are  the 
means  by  which  Governance  and  Strategic  Direction  are  translated  into  Results  and  Performance. 

3.1  Vision  of  a  Steady-state  DRDC  MAF 

The  DRDC  Management  Accountability  Framework  (MAF)  is  a  governance  tool  that 
facilitates  dialogue  between  managers  and  executive  leadership,  helping  to  clarify  managers  ’ 
understanding  of  their  roles  and  responsibilities  and  providing  a  framework  for  assessment 
and  improvement  of  business  processes. 

The  Process:  The  DRDC  MAF  process  will  complement  the  DRDC  business  cycle.  The  DRDC 
MAF  contains  1 0  elements  but  ordinarily,  RDEC  will  begin  a  MAF  cycle  by  determining  which 
elements  are  of  greatest  current  interest,  because  of  Agency  or  Departmental  priorities.  The 
resultant  MAF  cycle  will  focus  on  those  elements,  but  may  also  address  others  on  an  overview 
basis. 

The  MAF  process  will  collect  information  both  from  corporate  data  sources  and  from  a  survey  of 
DRDC  managers.  Corporate  staff  will  analyze  and  report  on  corporate  data  and  survey  results  at 
both  the  Agency  and  Centre  level,  in  the  context  of  the  elements  of  the  DRDC  MAF.  Results  of 
the  analysis  will  be  used  to  identify  both  areas  of  organizational  strengths  to  commend  and 
weaknesses  to  address,  including  identifying  areas  for  business  process  improvement.  Results 
would  also  support  the  commitments  of  the  ADM  (S&T)  to  the  DM  DND  and  TB  MAF. 
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Governance 

and 

Strategic 

Direction 

TheDRDC 
Charter,  the 
Defence 
Science  and 
Technology 
Strategy  and 
the  DND 
Business 
Planning 
process  ensure 
effective 
support  and 
functional  S&T 
authority  for 
Government, 
the  Canadian 
Forces,  and 
defence  and 
security 
partners 


Defence  R&D  Canada  Agency  Values 

Through  its  actions,  DRDC  Leadership  continually  reinforces  the  importance  of  the  agency's 
values  and  of  defence  ethics  in  the  delivery  of  results  to  Canadian  forces  and  defence  and 
security  partners. 


Policy  and  Programs 

DRDC  research  and  analytic 
capacity  is  developed  and 
sustained  to  assure  high 
quality  policy  options, 
program  design  and  advice  to 
DND/ CF.  Public  Safety 
Canada,  and  security 
partners. 


People 

DRDC  has  the  people,  work 
environment,  leadership  and 
capacity  to  provide  the  best 
in  science  and  technology 
for  defence  and  security, 
now  and  for  the  future 


S  &T  activities  are  client 
and  partner-centred,  policies 
and  programs  are 
developed  to  address  client 
and  partner  needs,  and 
partnerships  are 
encouraged  and  effectively 
managed. 


Risk  Management 

DRDC  prepares  for, 
manages  and  exploits 
risk  in  the  science  and 
technology  environment. 


Stewardship 

The  DRDC  Control 
regime  (assets,  money, 
people,  services,  etc.)  is 
integrated  and  effective, 
and  its  underlying 
principles  are  clear  to  all 
staff. 


Accountability 

Accountabilities  for  results 
are  clearly  assigned  and  are 
consistent  with  resources, 
and  delegations  are 
consistent  with  capabilities. 


Learning,  Innovation  and  Change  Management 

DRDC  fosters  creativity,  innovation,  and  change,  promotes  organizational  learning,  values 
corporate  knowledge,  and  learns  from  its  performance 


Results 

and 

Performance 


Relevant 
information  on 
results 
(internal, 
service  and 
program)  is 
gathered  and 
used  to  make 
decisions  in 
DRDC.  and 
agency  and 
public  reporting 
is  balanced, 
transparent, 
and  easy  to 
understand. 


Figure  4:  Expectations  Tier  of  DRDC  MAF 


The  Participants:  RDEC  will  establish  the  priorities  of  any  given  MAF  cycle. 

DSTEA^'^  will  be  the  OPl  for  the  DRDC  MAF.  DGSTO  and  DGRDCS  will  be  responsible  for 
provision  of  corporate  data,  and  DGMPRA  will  be  responsible  for  the  development  and  execution 
of  managers’  surveys  to  DSTEA  requirements.  Managers’  surveys  for  the  DRDC  MAF  would  be 
addressed  to  DRDC  Fevel  2  and  Fevel  3  Managers,  and  those  Fevel  4  functional  authorities  who 
are  members  of  Centre  management  committees.  DSTEA  would  analyze  and  report  the  results  of 
the  DRDC  MAF  cycle  to  RDEC. 

The  Resources:  DSTEA  will  plan  any  given  DRDC  MAF  cycle,  analyze  supplied  corporate  data 
and  the  results  of  the  managers’  survey,  and  report  the  results  to  RDEC  (17  person-weeks 
DSTEA  may  also  use  the  results  to  support  TB  MAF  requirements  (2  person-weeks). 

DGRDCS  and  DGSTO  will  provide  corporate  data  to  DSTEA  requirements  (5  person-weeks). 


The  organizational  designators  in  Section  3  are  familiar  to  the  client,  and  so  are  not  generally  written  in  full,  but  are 
defined  in  full  in  the  List  of  acronyms/initialisms,  page  71  of  this  report. 

These  resource  estimates  are  drawn  from  Section  4.2  of  this  report. 


14 


DRDC  Atlantic  CR  2009-136 


DGMPRA  will  develop  and  implement  a  managers’  survey  to  DSTEA  requirements  and  deliver 
survey  tabulations  to  DSTEA  (8  14  person-weeks). 

DRDC  Eevel  2  and  Eevel  3  managers,  and  those  Eevel  4  functional  authorities  who  are  members 
of  Centre  Management  Committees  will  be  required  to  complete  the  managers’  survey  (4  person- 
weeks,  based  on  1  hour  per  manager). 

3.2  The  DRDC  MAF,  Expectations  Tier 

There  are  four  pillar  elements  of  the  DRDC  MAF  {Defence  R&D  Canada  Agency  Values; 
Governance  and  Strategic  Direction:  Results  and  Performance;  and  Learning,  Innovation  and 
Change  Management)  and  six  core  elements  {Policy  and  Programs;  People;  Service;  Risk 
Management:  Stewardship:  and  Accountability).  The  two  classes  of  elements  have  different 
characteristics  and  roles.  The  core  elements  apply  to  everyone  in  the  organization,  but  are 
especially  relevant  to  line  managers,  whose  activities  determine  the  success  of  DRDC  relative  to 
MAF  measures.  In  effect,  the  core  elements  set  the  standards  for  the  performance  of  management. 
The  pillar  elements  set  the  strategic  environment  for  management. 

The  two  classes  of  elements  were  also  developed  differently.  The  pillar  elements  were  developed 
largely  based  upon  reference  to  the  TB  MAF  together  with  TBS,  DND  or  DRDC  documents. 

This  ensured  that  the  pillar  elements  reflected  strategic  intentions.  Development  of  the  core 
elements  included  significant  investigation  of  the  peer-reviewed  literature,  to  ensure  that  the 
measures  employed  would  reflect  state-of-the-art  thought  on  public  administration  and 
management. 

3.3  The  Pillar  Elements 

There  are  four  pillar  elements.  Defence  R&D  Canada  Agency  Values  set  the  very  character  of 
DRDC,  defining  the  type  of  environment  and  the  behaviour  and  attitudes  of  employees  that  best 
represent  the  organization.  Governance  and  Strategic  Direction  describes  the  strategic  policy 
basis  for  DRDC  management  activities,  ensuring  that  management  activities  respond  to  policy- 
based  needs.  Results  and  Performance  identifies  the  need  for  evidence-based  decision-making 
and  evaluation  of  program  performance.  Finally,  Learning,  Innovation  and  Change  Management 
describes  the  environment  in  which  managers  manage.  If  the  core  elements  control  how 
managers  manage,  then  the  pillar  elements  set  the  environment  in  which  they  manage.  The 
provision  of  that  environment  is  the  purview  of  DRDC’s  executive  leadership,  rather  than  of 
managers.  Thus,  whereas  the  core  elements  of  the  DRDC  MAF  focus  on  Eevel  3  and  4 
managers,  the  pillar  elements  should  focus  on  Eevel  1  and  2  managers,  whose  behaviour  and 
attitudes  set  the  standard  for  Eevel  3  and  4  managers.  Of  course,  Eevel  3  and  4  managers  cannot 
ignore  the  pillar  elements,  but  the  pillar  elements  represent  enablers  or  constraints,  rather  than 
measures  of  performance. 

The  development  of  the  pillar  elements  relied  heavily  upon  the  results  of  application  of  TB  MAF 
to  the  Department  of  National  Defence,  together  with  corporate  documents.  This  approach 
reflected  the  strategic  nature  of  the  pillar  elements  and  the  need  to  establish  clear  linkages  with 
the  superior  TB  MAF. 
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3.3.1  Defence  R&D  Canada  Agency  Values 

Through  its  actions,  DRDC  Leadership  continually  reinforces  the  importance  of  the 
agency ’s  values  and  of  defence  ethics  in  the  delivery  of  results  to  the  Canadian  Forces 
and  defence  and  security  partners. 

Defence  R&D  Canada’s  values,  together  with  defence  ethics  standards,  position  DRDC  to 
respond  to  changing  world  and  organizational  circumstances  where  management  by  prescriptive 
rule-sets  will  often  be  found  wanting.  They  represent  DND  and  DRDC’s  implementation  of  the 
Values  and  Ethics  Code  for  the  Public  Service"*^. 

Indicators  and  Measures  for  this  element  are  shown  in  Table  3. 


Table  3:  Indicators,  Measures  and  Instruments  for  Defence  R&D  Canada  Agency  Values 


Indicators 

Measures  and  Instruments 

Exemplary  conduct 

Agency  values  are  reflected  in  the  realities 
and  cultures  of  its  Centres 

Values-based  management  practices 

Staff  identify  with  DRDC  Values 

•  Values  and  ethics  exercises 

•  Focus  groups  or  surveys  of  staff 

•  Focus  groups  or  surveys  of  partners  and  clients 

Ethical  decision  making  supported  by  the  Defence  Ethics 
Program 

•  Defence  Ethics  training  statistics  from  PeopleSoft® 

•  DRDC  Ethics  Coordinator  activity  statistics 

Organizational  performance  is  evaluated  based  on  Defence 
Ethics  Program 

•  Review  of  Ethics  Coordinator  activities 

•  Focus  groups  or  surveys  of  staff 

For  the  most  part,  measures  can  be  assessed  using  existing  instruments.  Commitment  to  the 
Defence  Ethics  program  can  be  readily  inferred  by  examining  PeopleSoft®-based  training 
records;  for  example:  have  all  new  staff  received  defence  ethics  training  within  an  acceptable 
time  of  being  taken  on  strength  (TOS)?  Similarly,  examining  the  activities  of  the  DRDC  Ethics 
Coordinator"^^  would  offer  insight  into  the  ethical  health  of  DRDC.  The  2008  Public  Service 
Employee  Survey  includes  ethics-related  questions  and  with  over  500  respondents  for  DRDC"^"* 
should  offer  a  reasonable  representation  of  DRDC’s  ethical  health.  A  potentially  stronger 


""  In  Sections  3.3  and  3.4,  the  discussion  of  each  element  of  the  proposed  DRDC  MAP  will  open  with  the  statement  of 
Expectations,  italicized. 

Treasury  Board  of  Canada  Secretariat,  “Values  and  Ethics  Code  for  the  Public  Service,”  Public  Works  and 
Government  Services  Canada,  Ottawa:  2003. 

Currently  the  Director,  R&D  Business  Affairs. 

Treasury  Board  of  Canada  Secretariat,  “Organizational  Report  for  Department  of  National  Defence  Assistant  Deputy 
Minister  Science  and  Technology  Group,”  2008  Public  Service  Employee  Survey  Results,  http://www.tbs- 
sct.gc.ca/pses-saff/2008/results-resultats/res-eng.aspx?cd=&ol=03&o2=019&o3=0Q0&o4=0Q0&o5=0Q0#tphp 

accessed  28  May  2009. 
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instrument  is  the  Defence  Ethics  Survey,  delivered  every  three  years, although  past  versions 
have  only  be  addressed  to  DND  employees  and  CF  Members  with  addresses  on  the  DWAN/DIN. 


On  an  ongoing  basis,  ethics-related  instruments  might  be  expanded  to  include  workshops  or  focus 
groups,  although  these  two  are  perhaps  more  diagnostic  or  even  corrective  measures  than 
assessment  tools. 

3.3.2  Governance  and  Strategic  Direction 

The  DRDC  Charter,  the  Defence  Science  and  Technology  Strategy,  and  the  DND 
business  planning  process  ensure  effective  support  and  functional  S&T  authority  for 
Government,  the  Canadian  Forces,  and  defence  and  security  partners. 

This  statement  identifies  key  elements  -  the  ‘essential  conditions’  -  that  serve  to  ensure  effective 
support  and  position  ADM(S&T)  as  the  Department’s  functional  authority  for  defence  S&T.  The 
DRDC  Charter  establishes  the  raison  d’etre  of  the  organization,  its  relationship  to  DND,  the  CF 
and  stakeholders,  and  governance,  organization  and  accountability."^'^  The  Defence  S&T  Strategy 
provides  the  framework  for  delivery  of  S&T  in  the  defence  and  security  environment,  both  by 
DRDC  and  by  its  partners."^’  Finally,  DRDC’s  integration  in  the  DND  business  planning  process 
ensures  that  DRDC’s  activities  will  support  the  Strategic  Outcomes  of  DND’s  Program  Activity 
Architecture. 

Indicators  and  Measures  for  this  element  are  shown  in  Table  4. 


Most  recently  in  2007;  see  Fraser,  Kyle,  “The  2007  Defence  Ethics  Survey:  Summary  of  the  Overall  CF  and  DND 
Findings  for  Decision  Makers,”  Defence  R&D  Canada,  DRDC  CORA  TN  2008-016,  August  2008. 

Defence  Research  and  Development  Canada,  “Framework  Document,”  Approved  by  Treasury  Board  Ministers,  27 
July  2000. 

Department  of  National  Defence,  “Defence  S&T  Strategy:  Science  and  Technology  for  a  Secure  Canada,”  Ottawa, 
December  2006. 
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Table  4:  Indicators,  Measures  and  Instruments  for  Governance  and  Strategic  Direction 


Indicators 

Measures  and  Instruments 

Support  to  DM/DND  through  ADM(S&T) 

Confidence  of  ADM(S&T) 

Management  Framework  aligned  to 

Coherent  policy  agenda 

strategic  outcomes 

•  DRDC  Business  Plan  to  show  links  to  PAA 

•  DRDC  PGAs  to  show  coherence  with  sub-  and  sub- 

The  right  executive  team 

sub-activities  in  PAA 

Results-focused  corporate  priorities 

Strong  management  cadre 

.  Analysis  of  DRDC  PMAs''® 

Strategic  resource  allocation/reallocation 

•  DRDC  MAF  survey  for  management  cadre 

based  on  performance 

performance  against  core  elements 

Integrated  management  agenda 

Management  framework  supports  priority  setting,  reallocation, 
and  alignment  with  government-wide  priorities 

Horizontal  collaboration 

•  Business-line  planning  allows  alignment  of  PGA'*® 
resources  to  strategic  agenda 

Environmental  scanning 

•  Comparison  of  successive  Business  Plans  for 
evidence  of  priority-setting  and  reallocation 

•  Partner  Group  Agreements  (PGA)  form  the  basis  for 
results-focused  S&T 

•  Do  PGAs  reflect  Business  Plan  priorities? 

Organizational  engagement  in  administrative  and  science- 
based  horizontal  initiatives 

•  Evidence  of  activities  (S&T  ADMs,  CRTI®°,  etc.) 

Organizational  engagement  in  administrative  and  science- 
based  horizontal  initiatives 

•  DRDC’s  Technology  Watch  program  provides 
warning  of  disruptive  technologies 

3.3.3  Results  and  Performance 

Relevant  information  on  results  (internal,  service  and  program)  is  gathered  and  used  to 
make  decisions  in  DRDC,  and  departmental  and  public  reporting  is  balanced  and 
transparent,  and  easy  to  understand. 

This  statement  establishes  the  need  for  rational,  evidence-based  decision  making  and  for  timely, 
neutral  and  transparent  reporting  of  program  results. 

Indicators  and  Measures  for  this  Element  are  shown  in  Table  5. 


PMA:  Personal  Management  Agreement. 

PGA:  Partner  Group  Agreement. 

CRTI:  CBRN  (Chemical  Biological  Radiological  Nuclear)  Research  and  Technology  Initiative. 
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Table  5:  Indicators,  Measures  and  Instruments  for  Results  and  Performance 


Indicators 

Measures  and  Instruments 

Monitoring  and  reporting  of  program, 
service  and  internal  results 

Integrated  financial  and  non-financial 
performance  information  used  in  DRDC 
decision  making 

DRDC  reporting  based  on  measurable 
outcomes 

Benchmark  against  the  best 

Transparent,  timely  and  accessible 
communications 

Quality  of  DRDC  Business  Plan  and  Annual  Report 

•  Assessment  of  the  degree  to  which  these 
documents  support  superior  (RPP,  DPR®^)  or 
external  (transparency)  requirements 

Staff  and  Client  survey  results 

•  Public  Service  Employee  Survey,  Defence  Ethics 
Survey  (if  DRENet-enabled),  MAE  surveys 

Progress  in  strengthening  financial  and  program  results,  and 

performance  measurement 

•  Ongoing  tracking  of  available  measures  from 
sources  such  as  CPME“;  e.g.:  ongoing 
performance  against  project  milestones; 
expenditures  against  plans;  etc. 

DRDC  monitoring  and  review  of  performance 

•  This  reflects  the  activities  of  DSTEA  and  so  would 
warrant  program  evaluation  of  DSTEA  activities 

Risk-based  evaluation  plans  (reviewed  regularly)  and  follow¬ 
up 

•  Evidence  of  RDEC  decisions  on  past,  current  and 
future  requests  to  CRS®^  for  program  evaluations 

Performance  against  external  benchmarks 

•  Peer  reviews  of  scientific  programs 

•  Potentially,  evaluations  or  audits  of  corporate 
services  activities 

3.3.4  Learning,  Innovation  and  Change  Management 

DRDC  fosters  creativity,  innovation  and  change,  promotes  organizational  learning, 
values  corporate  knowledge,  and  learns  from  its  performance  (to  better  support  the 
defence  and  security  community). 

This  statement  establishes  the  environment  necessary  for  an  adaptive,  forward-looking 
organization. 

Indicators  and  Measures  for  this  Element  are  shown  in  Table  6,  overleaf 


RPP:  Report  on  Plans  and  Priorities;  DPR:  Departmental  Performance  Reports. 
CPME:  Collaborative  Program  Management  Environment. 

”  CRS:  Chief  Review  Services. 
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Table  6:  Indicators,  Measures  and  Instruments  for  Learning,  Innovation  and  Change 

Management 


Indicators 

Measures  and  Instruments 

A  learning  organization  with  strategic 
vision 

A  culture  that  anticipates  change,  values 
innovation,  and  supports  transformation 

An  organization  that  examines  its  own 
performance  to  identify  need  for  change 

An  organization  that  empowers  managers 
through  delegation 

An  organization  that  manages  knowledge 

Progress  and  investment  in  organizational  learning 

•  PeopleSoft®  provides  information  on  training  and 
development  activities 

•  FMAS®^'  provides  information  on  training  and 
development  direct  investment 

•  Corporate  initiatives  (such  as  EXPEDITION  09  1.4) 
provide  a  record  of  organizational  intent  and 
commitment 

•  A  DRDC  MAE  survey  would  address  employees’ 
perceptions  of  the  effectiveness  of  organizational 
learning  initiatives  and  practices 

Progress  in  knowledge  management 

•  CPME  will  provide  information  on  investment  and 
progress  on  research  in  knowledge  management 

•  Assessment  of  operational  application  of  knowledge 
management  will  require  a  variety  of  instruments, 
including: 

o  Review  of  CANDID®  over  time 

0  Review  of  PGAs 

o  DRDC  MAP  survey  questions 

Stakeholder/staff  perceptions  of  organizational  adaptability, 
change  and  innovation 

•  Client/partner  surveys 

•  Staff  surveys,  either  directly  as  DRDC  MAP  surveys 
or  potentially,  the  Public  Service  employee  Survey  if 
appropriate  questions  are  asked 

Linkages  between  organizational  performance  and 
organizational  development  initiatives 

•  Analysis  of  DRDC  Business  Plans,  PGAs  and 
EXPEDITION  XX  work  plans  to  establish  strength  of 
linkages. 

3.4  The  Core  Elements 

There  are  six  core  elements.  The  Policy  and  Programs  element  addresses  the  policy  bases  for 
DRDC  programs.  The  People  element  addresses  human  resources  management,  and  reflects  the 
critical  importance  of  human  capital  in  a  knowledge-based  organization  like  DRDC.  The  Service 
element  is  considerably  reduced  in  scope  from  the  TB  Citizen-centred  Service  element  since 
DRDC,  and  DND  for  that  matter,  is  not  generally  engaged  in  direct  services  to  citizens.  The  Risk 
Management  element  offers  an  interesting  challenge,  where  the  bureaucracy  at  large  largely  tries 
to  reduce  risk  whereas  S&T  organizations  often  try  to  exploit  risk.  The  Stewardship  element 
addresses  a  fundamental  aspect  of  public  service,  often  exemplified  as  ‘protecting  the  public 


FMAS:  DND’s  SAP®-based  Financial  Management  System. 
CANDID:  Canadian  Defence  Information  Database. 
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purse.  ’  Finally,  the  Accountability  element  addresses  the  crucial  features  of  hierarchical 
relationships  in  any  modern  organization. 

3.4.1  Policy  and  Programs 

DRDC  research  and  analytic  capacity  is  developed  and  sustained  to  assure  high  quality 
policy  options,  program  design  and  advice  to  DND/CF,  Public  Safety  Canada,  and  security 
partners. 

The  Policy  and  Programs  element  raises  the  oft-recurring  question  of  the  relationship  between 
science  and  policy  in  government,  and  the  potential  differences  between  ‘science  in  policy’  and 
‘policy  in  science.’ 

The  central  problem  of  Science  in  Policy  comes  out  of  Principal-Agent  theory^®.  This  theory, 
borrowed  from  economics,  is  predicated  on  the  relationship  between  two  actors  -the  principal  and 
the  agent.  The  principal  does  not  have  the  appropriate  resources  (such  as  scientific  knowledge)  to 
achieve  its  goals,  and  so  must  use  the  resources  it  does  have  (such  as  money)  to  contract  an  agent 
who  possesses  the  required  resources  to  achieve  the  goals  on  its  behalf  The  principals  in  this 
situation  are  DND/CF  and  Public  Safety  Canada  (PS),  while  the  agents  are  the  scientists  of 
DRDC. 

The  problem  that  arises  out  of  Principal-Agent  theory  is  that  of  information  asymmetry;  the 
principal  has  no  way  of  knowing  if  the  agent  is  providing  quality  outputs  because  the  principal, 
by  definition,  lacks  the  necessary  resources  to  judge  the  quality.  The  only  recourse  for  the 
principal  is  to  either  trust  the  agent  or  monitor  the  agent  in  an  effort  to  ensure  it  is  in  compliance 
with  the  contract.^*  In  the  context  of  government  and  scientists,  the  need  for  Governmental 
accountability  to  Parliament  for  spending  and  program  quality  requires  that  agents  be  monitored. 

Exacerbating  the  issue  of  information  asymmetry  is  the  tension  between  the  technocratic  and 
democratic  views  of  policymaking,  particularly  in  sciences.  Technocrats  believe  that  science 
and  technology  is  neutral  and  the  best  policy  options  will  be  generated  if  experts  are  allowed  to 
formulate  policy  based  on  their  exclusive  knowledge.  Democrats  believe  that  science  and 
technology  have  social  and  political  biases,  so  for  policy  to  be  successfril,  it  must  incorporate 
public  and  non-scientific  values.'^®  An  effective  agent-monitoring  framework  would  need  to 
balance  rational,  evidence-based  decision  making  with  democratic,  public  participation.'^' 

The  central  problem  of  Policy  in  Science  is  one  of  lack  of  consistency  in  program  evaluation,  or 
more  specifically,  a  lack  of  a  policy  framework  which  would  allow  one  to  answer  basic  questions 


David  H.  Guston,  “Principal-agent  theory  and  the  structure  of  science  policy,  revisited:  ‘science  in  policy’  and  the  US 
Report  on  Carcinogens S  Science  and  Public  Policy,  October  2003  30:5,  p.  347. 

Dietmar  Braun  and  David  H.  Guston,  “Principal-agent  theory  and  research  policy:  an  introduction”  Science  and 
Public  Policy,  October  2003  30:5,  p.  303. 

^*Ibid,p.  305. 

Sheila  Jassanoff  “Rationalizing  Politics”  in  The  Fifth  Branch:  Science  Advisors  as  Policymakers.  (Cambridge: 
Harvard  University  Press,  1994)  p.  15. 

^  Philip  L.  Bereano,  “Reflections  of  a  Participant-Observer:  The  Technocratic/Democratic  Contradiction  in  the  Practice 
of  Technology  Assessmenf’  Technological  Forecasting  and  Social  Change.  54:1997,  pp.  163-175. 

E.  Melanie  DuPuis  and  Brian  J.  Gareau,  “Neoliberal  Knowledge:  the  Decline  of  Technocracy  and  the  Weakening  of 
the  Montreal  Protocol”  Social  Science  Quarterly,  89:5,  December  2008,  p.  1214. 
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such  as  ‘does  funding  and  delivering  on  this  project  further  the  goals  for  government?’  and  ‘did 
this  project  meet  its  goals?’  The  lack  of  a  framework  also  creates  an  inability  to  compare  project 
outcomes  from  different  scientific  fields/^  Development  and  application  of  a  policy  framework 
that  addresses  these  issues  would  also  function  as  a  monitoring  mechanism  for  assessing 
accountability  in  the  contributions  of  science  to  policy  making.  An  appropriate  monitoring 
mechanism  can  provide  a  double-barrelled  solution  by  addressing  the  issues  of  both  Science  in 
Policy  and  Policy  in  Science. 

S&T  activity  outcomes  have  often  been  evaluated  at  the  Program  level,  according  to  overly  broad 
strategic  goals. This  approach  makes  it  difficult  to  determine  the  success  of  any  one  individual 
project  within  a  Program,  or  if  that  project  has  made  good  use  of  public  funding. 

One  promising  approach  to  measuring  and  evaluating  S&T  programs  is  to  borrow  from 
investment  management  concepts.  Science  and  technology  outcomes  are  assessed  from  a 
‘bottom-up’  perspective.  Each  project  is  evaluated  for  its  outcomes  or  ‘defined  investment  goal’ 
and  how  those  outcomes  support  the  Program  or  ‘portfolio’  goals.  Accountability  in  this  context 
includes  having  a  ‘balanced  portfolio’  of  both  high  risk  and  low  risk  S&T  projects.®"'  In  the 
vacuum  left  by  the  current  absence  of  an  accountability  approach  to  S&T®®,  utilizing  this  goal- 
driven,  bottom-up  approach  to  accountability  seems  prudent. 

Jackson  describes  portfolio  management: 

The  central  challenge  in  portfolio  design  is  the  selection  of  investments  so  their 
component  goals  will  combine  appropriately  to  achieve  the  desired  portfolio  goal. 
Without  clearly  articulated  goals,  it  is  impossible  to  assess  rationally  how  individual 
R&D  investments  fit  as  components  of  program  or  portfolio  goals.®® 

Notice  that  while  the  portfolio  concept  is  borrowed  from  the  financial  sector,  there  is  no  attempt 
to  attach  monetary  value  to  outcomes.  Instead,  the  outcomes  are  valued  according  to  whether  or 
not  they  have  achieved  their  goals.  By  articulating  the  desired  outcome  in  ‘units’,  we  can  select 
the  metric  which  is  appropriate  for  the  outcome;  the  specific  unit  metric  will  depend  on  the  goal. 
By  measuring  success  according  to  achievement  of  goals,  we  can  compare  success  rates  of 
projects  which  cannot  be  directly  compared  (for  example:  comparing  workforce  development  to 
knowledge  expansion). 

A  project  really  only  fails  when  it  does  not  meet  its  goal;  so  success  can  only  be  assessed  if  the 
goal  is  articulated.  Generally,  if  R&D  seeks  to  introduce  new  knowledge,  then  any  project  which 


Irwin  Feller,  “Mapping  the  frontiers  of  evaluation  of  public-sector  R&D  programs.”  Science  and  Public  Policy, 
December  2007,  34: 10. 

Brian  A.  Jackson,  “Federal  R&D:  Shaping  the  National  Investment  Portfolio”  in  Shaping  Science  and  Technology 
Policy:  The  Next  Generation  of  Research,  David  H  Guston  and  Daniel  Sarewitz,  eds.  (Madison:  University  of 
Wisconsin  Press,  2006),  pp.  33-54. 

'’hbid. 

Eleanor  Fast,  Mobilizing  Science  and  Technology:  The  New  Federal  Strategy,  Library  of  Parliament,  Science  and 
Technology  Division,  December  2007.  p.  2,  and  Industry  Canada,  Mobilizing  Science  and  Technology  to  Canada 's 
Advantage,  2007,  p.  88. 

^  Brian  A.  Jackson,  “Federal  R&D:  Shaping  the  National  Investment  Portfolio”  in  Shaping  Science  and  Technology 
Policy:  The  Next  Generation  of  Research,  David  H  Guston  and  Daniel  Sarewitz,  eds.  (Madison:  University  of 
Wisconsin  Press,  2006),  p.  38. 
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contributes  to  scientific  knowledge  can  be  considered  a  success,  even  if  it  is  not  necessarily 
‘applied  science.’ 


Jackson  gives  five  generalized  goals,  each  with  different  desired  outcomes: 

•  Expansion  of  the  body  of  knowledge 

•  Monetary  returns  via  economic  growth 

•  Mission  directed  needs  for  R&D  results 

•  Workforce  development  and  education 

•  Maintenance  of  national  scientific  and  technical  infrastructure  and  capacity 

These  generalized  goals  can  be  aligned  with  the  Advantage  Goals  of  the  Canadian  S&T 
Strategy as  in  Table  7: 


Table  7:  Jackson ’s  Goals  Aligned  to  the  Industry  Canada  S&T  Strategy 


S&T  Strategy 

Portfolio  Management 

Entrepreneurial  Advantage 

•  Monetary  returns  via  economic  growth 

•  Mission  directed  needs  for  R&D 
results 

Knowledge  Advantage 

•  Expansion  of  the  body  of  knowledge 

People  Advantage 

•  Workforce  development  and 
education 

•  Maintenance  of  national  scientific  and 
technical  infrastructure  and  capacity 

The  ‘roll  up’  nature  of  the  portfolio  paradigm  also  fits  well  with  the  roll  up  nature  of  an  Agency 
level  MAE. 

Policy  capacity  can  be  defined  as  “the  ability  to  marshal  the  necessary  resources  to  make 
intelligent  collective  choices  about  and  set  strategic  directions  for  the  allocation  of  scarce 
resources  to  public  ends”'^*  or  more  simply,  the  ability  to  consistently  create  and  implement 
policy  which  meets  its  intended  objectives. 

Policy  capacity  is  dependent  on  having  a  critical  mass  of  human  capital  trained  in  policy 
development,  implementation  and  evaluation,  as  well  as  a  policy  framework  or  process  in  which 
these  analysts  do  their  job,  and  a  culture  which  supports  good  policy  making.  Since  policy 
capacity  is  dependent  on  these  mutable  factors,  it  will  naturally  vary  over  time  and  place.®® 


Industry  Canada,  Mobilizing  Science  and  Technology  to  Canada’s  Advantage,  2007. 

Martin  Painter  and  Jon  Pierre.  “Unpacking  Policy  Capacity:  Issues  and  Themes”  in  Challenges  to  State  Policy 
Capacity.  Martin  Painter  and  Jon  Pierre,  eds.  (New  York:  Palgrave  Macmillan,  2005),  p.  2. 

Martin  Painter  and  Jon  Pierre,  p.  3. 
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The  Indicators  tier  for  Policy  and  Programs  reflects  the  three  key  indicators  needed  for  policy 
capacity:  people,  a  framework,  and  supportive  culture.  The  indicators  closely  resemble  those  of 
the  TB  MAP,  but  are  tailored  for  clarity  and  to  reflect  the  DRDC  environment  and  level  of 
management  utilizing  the  MAP. 

Indicators  and  Measures  for  this  element  are  shown  in  Table  8.  Note  that  the  measure  related  to 
human  resources  management  for  the  policy  community  will  also  be  addressed  through  the 
people  element.  This  type  of  ‘cross-linking’  is  quite  common  throughout  the  DRDC  and  the  TB 
MAPs. 


Table  8:  Indicators,  Measures  and  Instruments  for  Policy  and  Programs 


Indicators 


Measures  and  Instruments 


Sustained  analytic  capacity  and  culture  of 
consultation,  review  and  challenge 


Results  focused  policy  and  program 
agendas  aligned  with  government’s 
horizontal  priorities 


Social  Evaluation  of  Impacts 


Confidence  of  the  ADM(S&T),  clients  and 
partners 


Quality  of  Policies  and  Programs 

•  Audits 

•  Peer  reviews 

•  Access  to  Information  (ATI)  program  performance 

•  Environmental  assessment  program  performance 

Comprehensive  Human  Resource  Management  plan  for  the 
policy  community 

•  Audit  of  succession  planning 

•  Audit  of  recruitment  and  retention  demographics 

•  Effectiveness  of  student  employment  programs 

•  MAE  survey  of  managers 

Investments  in  policy  capacity  and  research  tools. 

•  Evidence  of  Research  Collaboration  through  CPME 
or  manager  surveys 

•  Audit  of  professional  development  programs 

•  Scientific  equipment  maintenance  and  replacement 
records 


3.4.2  People 

DRDC  has  the  people,  work  environment,  leadership  and  capacity  to  provide  the  best  in 
science  and  technology  for  defence  and  security,  now  and  for  the  future. 

The  People  element  of  the  MAP  is  in  a  sense  ubiquitous.  The  human  resources  management 
regime  is  largely  consistent  across  the  core  public  service  and  so  differences  between  the  DRDC 
and  TBS  elements  should  be  more  a  reflection  of  the  DRDC  vernacular  than  substantive. 

In  December  2005,  the  Government  of  Canada  fully  implemented  the  Public  Service 
Modernization  Act  (PSMA).  The  Act  was  created  to  improve  and  reform  the  Public  Service  in 
order  to  meet  the  demands  of  the  2D*  Century.  Factors  such  as  new  technology  and  globalization 
had  lead  to  increased  service  demands  and  heightened  expectations  from  citizens;  a  shift  was 
occurring  in  labour  relations,  leading  to  the  need  for  a  more  collaborative  labour  relations  model; 
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and  the  appointments  process  had  become  cumbersome  and  ineffectual,  necessitating  the  creation 
of  a  new,  streamlined  hiring  system  which  could  function  more  effectively  while  protecting  the 
core  principle  of  merit-based  hiring.™  The  Public  Service  Employment  Act  (PSEA)’'  provides 
managers  with  the  flexibility,  authorities  and  accountability  necessary  to  support  the  PSMA. 

The  Staffing  Management  Accountability  Frameworlfi^  (SMAF)  translates  the  PSEA  into  a  more 
accessible  working  model.  It  meshes  with  the  MAE  to  provide  clearer  direction  on  the  People 
Element.  The  SMAF  is  comprised  of  eleven  elements: 

•  Delegation  of  Staffing 

•  HR  Planning  and  Integration  with  Business  Planning 

•  Organizational  HR  Support  Systems 

•  Organizational  Accountability  for  Results 

•  Flexibility  and  Efficiency 

•  Merit 

•  Non-partisanship 

•  Representativeness 

•  Access 

•  Fairness 

•  Transparency 

Other,  non-legislative  documents  relevant  to  Human  Resource  Management  in  the  Public  Service 
are  the  Treasury  Board's  Values  and  Ethics  Code  for  the  Public  Service^^  and  Guidance  for 
Deputy  Ministers^'’ ,  as  well  as  the  Privy  Council  Office  Annual  Reporfi\ 

Succession  planning  is  a  crucial  element  of  HR  management  and  is  vital  to  sustaining  capacity  in 
policy  and  programs.^® 

Indicators  and  Measures  for  the  People  element  of  the  DRDC  MAE  are  based  upon  the  TB  MAE, 
together  with  the  elements  of  the  SMAF,  the  results  of  the  2006  DND  MAE  assessment,  and  the 
requirements  for  Round  1  of  the  TB  MAE.  Indicators  and  Measures  are  shown  in  Table  9, 
overleaf  Instruments  for  assessment  of  measures  include  the  Public  Service  Employee  Survey, 
for  which  raw  data  are  available  for  custom  analysis,  not  just  the  published  reports. 


Senator  Sharon  Carstairs,  speech,  Move  Second  Reading  of  Public  Service  Modernization  Act.  June  5,  2003. 
PSMA  Resource  Centre.  Key  Messages:  Public  Service  Employment  Act.  http://www.psagency- 
agencefp.gc.ca/arc/hrmm-mgrh/psma-lmfp/centre/products-produits/messages_e.asp#PSEA. 
http://www.psc-cfp.gc.ca/plcv-pltq/frame-cadre/acco-resD/smaf-crgd-eng.htm. 

Values  and  Ethics  Code  for  the  Public  Service. 

Guidance  for  Deputy  Ministers  http://www.pco- 

bcp.gc.ca/index.asp?lang=eng&page=information&sub=publications&doc=gdm-gsm/doc-eng.htm. 

Privy  Council  Office  Annual  Report  http://www.pco-bcp.gc.ca/index.asp?lang=eng&page=clerk- 
greffier&sub=reports-rapports&doc=reports-rapports-eng.htm. 

Treasury  Board  of  Canada  Secretariat.  “Succession  Planning  and  Management  Guide.”  httD://www.tbs- 
sct.gc.ca/gui/sure-eng.asp. 
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Indicators 


Table  9:  Indicators,  Measures  and  Instruments  for  People 

Measures  and  Instruments 


Workforce  is  sufficient  in  numbers  and 
abilities  while  reflecting  the  Canadian 
population 


Comprehensive  HR  Strategic  plan  in  place 

•  DRDC  Human  Resources  Plan^^ 

•  PeopleSoft® 


Workplace  is  a  supportive  and  respectful 
environment,  including  respect  for  official 
language  requirements 


All  levels  of  management  demonstrate 
effective  leadership 


Progress  against  HR  targets  including  quality  of  work  life, 
official  languages,  employment  equity 

•  PeopleSoft® 

•  PS  Employee  Survey 

•  Employee  Assistance  Program  (EAP)  reports 

•  Progress  against  demographic  targets,  employment 

equity  staffing  activities,  language  of  work  complaint 
or  discrimination  statistics 


Employees  are  engaged  in  their  work  and 
have  opportunities  to  learn  and  grow 


Progress  in  measuring  &  improving  employee  engagement 

•  Recognition  &  reward  program  statistics 

•  Development  program  participation  levels 

•  Personal  Learning  Plan  statistics,  especially  goal 
achievement 


Quality  of  leadership 

•  Performance  Management  Program  statistics 

•  Public  Service  Employee  Survey 

Quality  of  labour  relations 

•  PS  Employment  Survey 

•  LMCC  and  LMRC  activities 

•  Alternative  Dispute  Resolution  statistics 

•  Classification  and  other  grievance  statistics 


3.4.3  Service 

S&T  activities  are  client  and  partner-centred,  policies  and  programs  are  developed  to 
address  client  and  partner  needs,  and  partnerships  are  encouraged  and  effectively  managed. 

The  TBS  document  Quality  and  Affordable  Service  for  Canadians:  Establishing  Service 
Standards  in  the  Federal  Governmenf^  v/as  implemented  in  1994.  The  TBS  service  standards 
stress  quality  and  affordable  services  at  the  citizen-public  servant  interface,  reflecting  the  fact  that 
most  government  departments  offer  services  directly  to  Canadians.  DRDC  is  an  exception  to  this, 
since  its  direct  services  are  delivered  to  DND,  other  government  departments  and  private 
enterprises. 


’’’’  For  example,  Defence  R&D  Canada,  “Human  Resources  Plan,  FY  2008-2009,”  undated. 

“Quality  and  Affordable  Service  for  Canadians:  Establishing  Service  Standards  in  the  Federal  Government  -  An 
Overview,”  httri://www.tbs-sct.gc.ca/pubs  pol/opepubs/TB  D3/OOUA-eng.asp. 
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Most  service-oriented  literature  and  research  focuses  on  private  sector  organizations  delivering 
direct  customer  service;  however,  the  conclusions  and  best  practices  taken  from  these  studies  are 
relevant  to  the  public  sector,  and  are  readily  transferable  to  the  DRDC  service  model  in  which 
customers  are  replaced  by  client  or  partner  organizations. 

The  goal  of  service  delivery  is  to  satisfy  the  recipients  of  the  service.  Client  satisfaction  is 
dependent  not  only  on  service  quality  but  also  on  the  client’s  perceived  value  of  that  service^®; 
this  view  is  echoed  in  the  very  title  of  the  TBS  service  standards  document  Quality  and 
Affordable  Service  for  Canadians.  There  is  evidence  that  high  satisfaction  levels  lead  to  increased 
client  retention  and  increased  regard  for  the  corporate  image.*® 

Service  quality  is  a  construct  encapsulating  a  number  of  dimensions.  Most  construct  models,  as 
demonstrated  in  Ghobadian  et  ah,  *'  can  be  aligned  to  group  the  various  dimensions  into  five 
broad  categories,  best  identified  by  Zeithaml  et  ah,  as  Tangibles,  Reliability,  Responsiveness, 
Assurance,  and  Empathy.*^  The  consumer  or  client  judges  the  quality  of  the  service  provided 
based  on  his  or  her  perception  of  these  dimensions,  which  may  not  necessarily  be  the  reality  of 
the  dimension  in  the  organization;  this  is  an  important  distinction.  For  example,  while  the 
organization  may  in  fact  have  cutting  edge  information  security,  the  client  may  not  perceive  this. 
These  dimensions  are  designed  to  apply  to  a  direct  client-provider  transaction;  however,  they  can 
equally  be  applied  to  DRDC’s  delivery  of  services  and  outputs  to  government  departments  and 
private  enterprises. 

Various  authors  observe  that  evaluation  of  service  quality  is  difficult  since  the  quality  cannot  be 
measured  directly  (as,  for  example,  a  failure  rate  or  some  other  technical  measure)  but  must  be 
based  on  user  experience  or  expectation.  **  It  can  be  difficult  for  an  organization  to  assess  or 
anticipate  a  client’s  perceived  value  of  a  service  because  each  individual  client  will  perceive 
value  relative  to  his/her  own  criteria.*"'  Generally,  clients’  perceived  value  of  a  service  will  be 
based  on  a  “trade-off  between  the  quality  or  benefits  [i.e.  utility]  they  perceive  in  product  relative 
to  the  sacrifice  they  perceived  by  paying  the  price.”*^  The  cost  or  ‘price’  of  a  service  can  include 
money,  time,  opportunity  costs  and  other  resources.  Because  the  organization  is  limited  in  its 
ability  to  impact  service  costs,  the  most  effective  method  for  influencing  perceived  value  is  by 
maintaining  high  service  quality. 

The  importance  of  perception  is  reinforced  in  an  earlier  study  by  Brown  and  Swartz,  which 
explored  professional  service  quality  in  primary  health  care  by  examining  client-professional 
relationships  between  doctors  and  patients.  *®  Brown  and  Swartz  studied  the  gaps  between  client 
expectations  and  experience,  and  between  client  expectations  and  professional  perceptions  of 


Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  “Relationships  and  impacts  of  service  quality, 
perceived  value,  customer  satisfaction  and  image:  an  empirical  study”.  The  Service  Industries  Journal,  29:2  Feb  2009, 

p.  121. 

Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  p.  122. 

**  Abby  Ghobadian,  Simon  Speller  and  Matthew  Jones,  “Service  Quality:  Concepts  and  Models.”  International  Journal 
of  Quality  &  Reliability  Management.  1 1 :9,  1994,  pp.  53-54. 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  “Delivering  Quality  Service,”  (New  York:  Free  Press, 
1990),  p.  26. 

For  example,  Ghobadian,  Speller  and  Jones  (1994),  or  Stephen  W.  Brown  and  Theresa  A.  Swartz,  “A  Gap  Analysis 
of  Professional  Service  Quality,”  The  Journal  of  Marketing,  Vol.  53,  No.  2  (Apr.,  1989),  pp.  92-98. 

Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  p.  1 14. 

Ibid. 

Stephen  W.  Brown  and  Theresa  A.  Swartz,  pp.  92-98. 
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those  expectations.  They  observe  the  need  either  to  align  service  provider  behaviours  and 
expectations  with  those  of  clients,  or  to  alter  client  expectations  and  experiences,  often  by 
education  on  the  subject  of  professional  capabilities  and  services. 

Measuring  service  quality  in  an  S&T  environment  is  more  complex  than  the  standards  illustrated 
by  the  Treasury  Board  Secretariat.  Setting  service  standards,  for  example,  may  be  straight 
forward  in  a  traditional,  client  centered  transaction,  such  as  Western  Economic  Diversification 
Canada  (WEDC),  where  “[rjesponse  to  initial  application  in  four  days”*^  is  presumably  a 
reasonable  and  measurable  service  standard,  given  a  90  percent  compliance  rate;  however,  the 
same  is  not  the  case  for  professional  services  (like  the  S&T  services  of  DRDC)  where  customer, 
client  or  partner  expectations  are  not  so  easily  quantifiable. 

Indicators  and  Measures  for  the  Service  element  are  given  in  Table  10.  The  DRDC  link  to  the  TB 
Measures  and  Indicators  can  be  seen,  after  accounting  for  the  DRDC  vernacular  and  reduction  in 
scope  for  the  DRDC  element,  through  deletion  of ‘citizen  focus.’ 


Table  10:  Indicators,  Measures  and  Instruments  for  Service 


Indicators 

Measures  and  Instruments 

Monitored,  continuously-improved  service 
quality 

Leading  advice  on  technology  exploitation 

Proactive  engagement  of  clients  and 
partners 

Effective  relationships  with  clients  and 
partners 

Service  improvement  and  transformation  planning  is  ongoing 

•  EXPEDITION  XX  ‘dashboard’  and  progress  reports 

•  DRDC  Annual  Report 

•  MAE  surveys  to  measure  cultural  adoption 

Client  &  partner  satisfaction  is  measured  annually  against 
standards 

•  Client  surveys 

•  Overview  Group  records 

•  MAP  surveys  (DRDC  manager  perceptions) 

Collaboration  is  key  to  delivering  DRDC  services 

•  DRDC  Annual  Report(Annex  on  S&T  Collaboration) 

•  Availability  of  DRDC-wide  SOPs 

•  Partner  and  client-focused  surveys 

•  MAP  surveys  (Collaboration) 

DRDC  makes  S&T  information  readily  available  to 
stakeholders 

•  DRDKIM®®  records  (CANDID  statistics) 

•  Partner  and  client-focused  surveys 

Treasury  Board  Secretariat,  “Quality  and  Affordable  Service  for  Canadians:  Establishing  Service  standards  in  the 
Federal  Government  -  An  Overview,”  at  httD://www.tbs-sct.gc.ca/pubs  poFonepubs/TB  D3/OOUA-eng.asp. 

**  DRDKIM:  Director  R&D  Knowledge  and  Information  Management 
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3.4.4  Risk  Management 

DRDC  prepares  for,  manages  and  exploits  risk  in  the  science  and  technology  environment. 

Public  sector  science  and  technology  organizations  like  DRDC  deal  with  a  central  conundrum: 
exploiting  risk  is  essential  to  innovation,  whereas  reducing  risk  is  fundamental  to  bureaucracies. 
The  DRDC  Expectations  statement  for  Risk  Management  speaks  to  this  conundrum  by  calling 
upon  DRDC  to  address  both  facets  of  it. 

Indicators  and  Measures  for  the  Risk  Management  element  are  largely  based  upon  those  of  the 
TB  MAE,  and  so  are  apparently  silent  on  the  subject  of  exploiting  risk.  This  need  not  be  so. 
DRDC’s  mission  establishes  the  agency  itself  as  a  risk  reduction  measure  in  the  international 
defence  and  security  context;  therefore,  DRDC  could  incorporate  risk  as  opportunity  concepts 
into  its  own  Corporate  Risk  Profile  (CRP),  as  long  as  the  DRDC  document  were  not  in  absolute 
conflict  with  DND’s  CRP,  as  yet  unpublished.  Thus,  alignment  of  the  DRDC  MAE  to  the  TB 
MAE  would  be  preserved,  with  risk  as  opportunity  being  addressed  at  the  DRDC  policy  level. 
Defence  R&D  Canada  might  also  address  risk  as  opportunity  concepts  through  customized 
training  as  suggested  by  Quigley  in  the  Annex  to  this  report. 

It  should  be  noted  that  Risk  Management  is  not  an  element  in  which  DND  performed  particularly 
well  in  the  TB  MAE,  receiving  an  “Opportunity  for  Improvement”  rating  in  Round  V,  in  which 
the  TBS  report  said: 

“DND  has  made  strong  progress  in  the  implementation  of  integrated  risk  management.  The 
department  has  identified  its  major  risks  to  the  successful  delivery  of  the  Defence  programme 
and  intends  to  integrate  risk  in  the  upcoming  planning  cycle.  While  risk  mitigation  strategies 
have  been  identified,  DND  has  yet  to  assign  monitoring  and  implementation  of  these 
strategies  to  senior  management.  TBS  is  encouraged  by  the  Department's  efforts  to  date  and 
believes  it  could  be  an  exemplary  federal  organization  in  this  area  of  management  once  its 
risk  management  system  is  complete  and  fully  functioning.”^^ 

This  weakness  in  Departmental  process  adds  further  complexity  to  risk  management  in  DRDC, 
and  to  the  tension  between  risk  as  opportunity  and  risk  as  failure. 

Indicators  and  Measures  for  Risk  Management  are  shown  in  Table  11,  overleaf 


**  DND  Assessment,  Round  V,  Available  at:  httD://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/2007/dnd/dnd- 
eng.asp. 
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Table  11:  Indicators,  Measures  and  Instruments  for  Risk  Management 


Indicators 

Measures  and  Instruments 

Risk  is  managed  within  the  context  of  the 

Risk  Management  is  integrated  with  decision  making, 

DND  Corporate  Risk  Profile 

planning  and  reporting 

Risk  Management  is  integrated  with 

•  Evidence  of  risk  management  processes  in  project 
documentation  and  CPME 

Risk  Management  tools,  training,  and  support  are  available 

planning  and  operations  structures 

for  staff 

throughout  all  levels  of  the  Agency 

•  PeopleSoft  training  statistics 

Capacity  exists  to  support  a  Risk 

•  Risk  management  SOPs,  etc. 

Management  demonstrates  leadership  in  Risk  Management 

•  Management  surveys 

Management  Culture 

Risk  Management  approach  compliments  DND  corporate  risk 
profile 

•  DRDC  Business  Plan  and  Annual  Report  alignment 
with  Corporate  Risk  profile  (DND  and  DRDC) 

•  Use  of  best  practices  repository 

Engagement  of  external  stakeholders  in 
assessing/communicating  risks. 

•  Review  of  TDP  documentation  and  associated 

Senior  Review  board  minutes 

3.4.5  Stewardship 

The  DRDC  control  regime  (assets,  money,  people,  services,  etc.)  is  integrated  and  effective, 
and  its  underlying  principles  are  clear  to  all  staff. 

The  term  ‘stewardship’  is  defined  as  “the  careful  and  responsible  management  of  something 
entrusted  to  one's  care.”^°  In  the  public  sector,  government  departments  act  as  stewards  of  the 
resources  of  Canadians;  national  resources  include  money  (specifically  tax  revenue),  information, 
environmental/  natural  resources,  national  reputation,  people,  and  physical  assets. 

The  Stewardship  element  of  the  Treasury  Board  MAP  seeks  to  place  controls  on  how  government 
employees  manage  resources.  Similar  to  the  problem  discussed  in  the  Policy  and  Programs 
element,  the  central  issue  in  Stewardship  is  based  on  Agency  Theory.  This  theory,  borrowed  from 
economics,  is  predicated  on  the  relationship  between  two  actors:  the  principal  and  the  agent.  The 
principle  is  the  party  which  ‘owns’  the  resource,  while  the  agent  is  the  party  responsible  for 
managing  the  resource.  The  principals  in  this  situation  are  Canadian  taxpayers,  through 
Parliament  and  Government,  while  the  agents  are  the  government  departments  and  employees. 
There  is  no  agency  problem  when  the  interests  of  the  principal  and  agent  coincide  and  each  party 
seeks  to  maximize  the  utility  of  the  resource;  however,  an  agency  problem  exists  when  the 


Merriam- Webster  Online  Dictionary,  http://www.merriam-webster.com/dictionarv/stewardship. 
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interests  of  the  principal  and  the  agent  diverge.^'  Because  the  agent  has  control  of  the  resources, 
s/he  can  utilize  the  resources  to  his/her  benefit,  resulting  in  agency  costs;  the  principal  can  reduce 
these  costs  by  imposing  internal  controls  on  the  agent,  such  as  audit  requirements,  reporting 
structures,  and  evaluations.^^ 

Controls  placed  on  agents  need  to  be  sufficient  to  curb  agency  costs  without  being  overly  costly 
or  cumbersome.  An  overly  rigid  control  regime  not  only  utilizes  excessive  resources  in 
comparison  to  the  potential  for  abuse  of  resources,  it  can  serve  to  disengage  employees  and  build 
resentment  towards  the  organization,  leading  to  diminished  motivation,  and  loss  of  trust. 

An  alternative  theory  may  offer  valuable  information  to  help  enhance  trust  in  the  organization. 
Ironically  named  Stewardship  Theory,  it  seeks  to  explain  why  it  is  that  an  agent  will  sometimes 
act  in  the  interests  of  the  principal  even  when  to  do  so  appears  counter  to  the  agent’s  own 
benefit.®"^ 

Stewardship  Theory,  also  rooted  in  economic  theory,  describes  the  relationship  between  the 
principal  -  Canadian  taxpayers  through  Parliament  and  Government  -  and  the  stewards- 
departments  and  public  servants.  Just  as  in  Agency  theory,  each  party  seeks  to  maximize  the 
utility  of  resources  to  his/her  own  benefit.  The  steward  has  control  of  the  principal’s  resources, 
and  will  utilize  the  resources  to  maximize  his/her  own  utility,  which  may  result  in  agency  costs  if 
the  steward’s  interests  are  not  aligned  with  those  of  the  principal.  Stewardship  Theory  postulates 
that  the  steward  places  higher  value  on  cooperative  behaviours  because  this  will  provide  greater 
utility  than  self-serving  behaviours. The  principal  can  encourage  cooperative  behaviours  by 
granting  the  steward  a  degree  of  trust;  this  will  lead  to  intrinsic  motivation  to  achieve  the 
organization’s  goals,  based  on  the  steward’s  desire  for  higher  order  needs  such  as  self- 
determination,  achievement,  and  personal  growth. 

Indicators  and  Measures  for  Stewardship  are  shown  in  Table  12,  overleaf  The  indicators  are 
broadly  based  upon  those  of  the  TB  MAP,  but  revised  to  reflect  the  DRDC  vernacular,  as  are  the 
Measures. 


James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  “Toward  a  Stewardship  Theory  of  Management”, 
Academy  of  Management  Review,  22:1,  1997,  p.  22;  Dietmar  Braun  and  David  H.  Guston,  “Principal-agent  theory  and 
research  policy:  an  introduction”  Science  and  Public  Policy,  October  2003  30:5. 

James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  22. 

Jens  Grundei,  “Chapter  3:  Examining  the  Relationship  between  Trust  and  Control  in  Organizational  Design”  in 
Organization  Design:  the  Evolving  State  of  the  Art,  R.M.  Burton,  B.  Eriksen,  D.D.  Hakonsson,  C.C.  Snow,  eds. 
Springer  Information  and  Organization  Design  Series,  6,  2006,  52;  James  H.  Davis,  F.  David  Schoorman,  and  Lex 
Donaldson,  “Toward  a  Stewardship  Theory  of  Management”,  p.  25. 

James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  24. 

Ibid. 

Jens  Grundei,  p.  52;  James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  37. 
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Indicators 


Table  12:  Indicators,  Measures  and  Instruments  for  Stewardship 

I  Measures  and  Instruments 


Management  systems  provide  relevant 
information  and  effective  feedback  on 
resources,  results  and  controls 


Reports  and  records  are  timely  and  accurate 

•  Financial  forecast  performance 

•  Managers’  survey 


Effective,  efficient  and  economical  project 
management 


Established  internal  audit  function 


Expenditures  are  within  budget 

•  FMAS  and  forecast  performance 

Resources  are  procured  and  used  appropriately  to  achieve 
goals 

•  Procurement  file  performance 

•  Managers’  survey 


Compliance  with  policies,  regulations,  and  Assets  are  inventoried  and  strategically  managed 
legislation  ,  Frequency  of  asset  sighting 

•  Write-offs  or  transfers 


Information  is  utilized  and  protected  appropriately 

•  Security  infraction  statistics  (if  releasable) 

•  Protection  of  personnel  files 

•  Protection  of  classified  files/assets 


Deliverables  are  timely  and  effective 
•  Program  audits 
.  CPME 


3.4.6  Accountability 

Accountabilities  for  results  are  clearly  assigned  and  are  consistent  with  resources,  and 
delegations  are  consistent  with  capabilities. 

Aucoin  and  Heintzman  identify  three  purposes  of  accountability  in  a  bureaucratic  system:  control, 
assurance  and  continuous  learning.^’  Accountability  is  a  mechanism  for  exercising  control  over 
the  decision  making  of  subordinates  in  a  system  of  devolved  authority.  Managing  for  and 
reporting  outcomes  and  results  helps  ensure  accountability,  as  these  increase  transparency  and 
disclosure  without  resorting  to  excessive  micro-management. 

In  the  parlance  of  agency  theory^^,  accountability  also  functions  to  “provide  assurance  to 
principals  that  their  agents  are  fulfilling  their  responsibilities  as  intended.”''*®  To  provide 
assurance,  accountability  regimes  must  specify  “goals  and  objectives,  service  entitlements  and 


Peter  Aucoin  and  Ralph  Heintzman,  “The  dialectics  of  accountability  for  performance  in  public  management 
reform”,  International  Review  of  Administrative  Sciences,  66,  2000,  48,  p.  45. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  49. 

Discussed  previously  in  Policy  and  Programs  Element,  page  21  of  this  report. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  49. 
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standards,  best  management  praetiees  and  performance  targets”’*”  and  include  systematic 
auditing,  inspection  and  review.  A  key  principal  of  an  effective  accountability  structure  is  that 
while  “individuals,  both  when  acting  alone  or  in  collaboration,  must  be  held  accountable  for  their 
individual  performance,”’*’^  there  is  also  an  acknowledgement  of  “the  constraints  that  affect  the 
realization  of  outputs  and  outcomes  and  over  which  public  servants  have  little  or  no  control.”’*’^ 
This  is  particularly  important  when  determining  responsibility  for  events  and  outcomes. 

To  lessen  the  vulnerability  of  each  person  in  the  superior-subordinate  relationship,  it  is  essential 
to  have  objective  measures  of  performance,  so  that  the  subordinate  is  aware  of  the  standard  to 
which  his/her  performance  will  be  held  accountable,  and  the  manager  has  evidence  of  the 
subordinate  having  met  (or  not)  his/her  responsibilities  for  performance.  ’*’"' 

Demonstration  of  performance  against  expectations  implies  a  regime  of  reporting  and  review:  the 
subordinate  reports  to  the  superior  and  the  superior  reviews  performance.  But  it  is  not  enough  to 
just  review  the  performance  of  subordinates.  To  have  a  true  accountability  regime,  the  superior 
must  utilize  the  information  gathered  from  the  review  to  benefit  the  organization,  otherwise,  no 
real  purpose  is  served  by  the  accountability  structure.  According  to  Aucoin  and  Heintzman, 

“those  responsible  for  managing  the  managers. . .  [must]  ensure  that  accountability  for  individual 
management  performance  counts”.  ’*’^ 

In  addition  to  focusing  on  control  and  assurance,  the  goal  of  an  accountability  regime  is  also  to 
support  continuous  improvement  in  policy,  the  organization  or  its  management.’*”’  Results  of 
individual  performance  assessments  (both  positive  and  negative)  should  be  utilized  as  learning 

•  •  107 

opportunities. 

Elliot  Jaques  describes  the  accountability  regime  as  a  three  level  managerial  linkage  where  “the 
manager  is  accountable  to  the  manager-once-removed  for  his  own  work  and  for  the  work  of  his 
subordinate,  and  he  manages  his  subordinate  within  the  terms  of  reference  set  by  the  manager- 
once-removed.”’*’*  The  manager-once-removed  is,  in  turn,  accountable  to  the  Governing  Body  for 
his  own  work  and  for  the  work  of  his  subordinate,  the  manager.  ’*’®  The  manager  may  delegate 
responsibilities  to  subordinates  (in  fact,  ordinarily  must  do  so)  but  cannot  escape  accountability  to 
his  own  manager  (the  manager-once-removed)  for  those  delegated  responsibilities  and  authorities. 
The  manager  must  have  requisite  delegated  authority  from  the  manager-once-removed  to  exploit 
the  resources  assigned,  whether  human  or  financial.  It  has  been  argued  that  “accountability  should 
be  viewed  as  that  which  is  exchanged  for  the  devolution  of  authority.  It  can  be  a  positive 


Ibid. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  50. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  54. 

Elliot  Jaques,  “Chapter  Four:  Managerial  Accountability,  Authority,  and  Dependence”  in  A  General  Theory  of 
Bureaucracy.  (New  York:  Halsted  Press.  1976),  p.  79. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  5 1 . 

Peter  Aucoin  and  Ralph  Heintzman,  p.  52. 

Peter  Aucoin  and  Mark  D.  Jarvis,  Modernizing  Government  Accountability:  A  Framework  for  Reform  (Ottawa: 
Canada  School  of  Public  Service,  2005),  p.  9. 

Elliot  Jaques,  p  65. 

’*’^Ibid. 
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opportunity  to  demonstrate  performance  results  against  the  expectations  of  those  who  have  faith 
and  trust  in  an  individual,  an  element  of  the  organization,  or  the  organization  itself.”"® 

Frameworks  and  systems  describe  the  environments  in  which  accountability  works,  or  is  required, 
but  do  not  define  accountability  per  se.  Within  the  bureaucratic  system,  the  definition  of 
accountability  provided  by  Aucoin,  Smith,  and  Dinsdale  eloquently  incorporates  the  aspects  of 
accountability  that  are  necessary  for  good  governance: 

Accountability  [is]  to  justify,  explain  or  defend  one’s  actions  (or  those  of  one’s 
subordinates)  based  on  powers  and  responsibilities  bestowed  by  a  superior 
authority.  The  account  may  encompass  a  statement  of  any  necessary  corrective 
action  to  be  taken.  The  superior  authority  has  the  obligation  to  hold  to  account  all 
those  on  whom  it  has  bestowed  powers  and  responsibilities.'" 

Responsibility  is  a  key  feature  of  this  definition  of  accountability  and  it  is  important  to  note  again 
that  responsibility  is  not  synonymous  with  accountability.  Rather,  consider  that  responsibilities 
are  delegated  to  subordinates  along  with  the  authority  necessary  to  reasonably  perform  these 
delegations;  in  exchange  the  subordinate  must  account  to  his/her  superior  for  his/  her 
performance  of  these  responsibilities.  The  Somalia  Inquiry  attempted,  without  much  success,  to 
clarify  the  difference  as  follows: 

One  who  is  authorized  to  act  or  exercise  authority  is  ‘responsible.’  Responsible 
officials  are  held  to  account.  An  individual  who  exercises  powers  while  acting  in 
the  discharge  of  official  functions  is  responsible  for  the  proper  powers  or  duties 
assigned."^ 

This  murky  explanation  offered  in  Dishonoured  Legacy  is  unfortunate  given  the  importance  of 
the  inquiry  to  the  organizational  development  of  the  Department  of  National  Defence  and  the 
Canadian  Forces. 

A  more  useful  expression  is  found  in  Organization  and  Accountability,  where  the  Department  of 
National  Defence  solidly  links  the  triad  of  responsibility,  authority  and  accountability,  while 
introducing  the  concept  that  delegated  authorities  and  responsibilities  bring  with  them  the  implicit 
obligation  to  act: 

Flaving  a  [delegated]  responsibility  involves  having  the  [delegated]  authority  and 
the  obligation  to  act,  including  the  authority  to  direct  or  authorize  others  to  act.  It 
also  means  being  accountable  for  how  those  responsibilities  have  been  carried 
out  in  light  of  agreed  or  set  expectations,  particular  duties,  or  obligations.  In  a 
public  sector  organization  . . .  each  individual  is  obliged  to  account  fully  and 
promptly  to  those  who,  in  the  hierarchy,  conferred  the  responsibilities,  for  the 


"®  “Strengthening  Accountability  and  Comptrollership  in  National  Defence,”  Department  of  National  Defence,  16  June 
2004,  C8/22. 

' ' '  Peter  Aucoin,  Jennifer  Smith,  and  Geoff  Dinsdale,  Responsible  Government:  Clarifying  essentials,  dispelling  myths 

and  exploring  change,  (Ottawa:  Canadian  Centre  for  Management  Development,  2004),  p.  88. 

112 

Canada,  Report  of  the  Commission  of  Inquiry  into  the  Deployment  of  Canadian  Forces  to  Somalia,  Dishonoured 
Legacy:  The  Lessons  of  the  Somalia  Affair,  (Ottawa:  Public  Works  and  Government  services  Canada,  1997),  p.  393. 
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way  they  have  been  carried  out  and  for  how  the  relevant  authorities  have  been 
used."^ 


The  present  authors  have  added  “[delegated]”  to  the  quotation  to  stress  that  responsibility  and 
authority  are  typically  bestowed  by  a  superior,  who  will  hold  the  actions  of  the  delegate  to 
account,  as  per  Aucoin,  Smith  and  Dinsdale. 

The  accountability  element  of  the  proposed  DRDC  MAP  will  use  Jaques’  accountability  regime, 
along  with  Aucoin  and  Heinzman’s  tri-partite  explanation  of  accountability’s  purposes,  as  its 
foundation.  The  DRDC  MAP  will  focus  upon  the  superior-subordinate  relationship,  to  support 
both  reporting  and  review  and  continuous  process  improvement.  Indicators  and  measures  for 
Accountability  are  shown  in  Table  13. 


Table  13:  Indicators,  Measures  and  Instruments  for  Accountability 


Indicators 


Measures  and  Instruments 


Clear  accountabilities  and  responsibilities 
for  due  process  and  results 


Delegations  consistent  with  capabilities 


Management  goals  are  identifiable  and  measureable 

•  Performance  management  program  (PMP)  for  Level 
2  Managers 

•  Management/annual  objectives  for  Levels  3  and  4 
(assessment  of  local  practice) 

•  Managers’  survey 

•  Subordinate’s  Personal  Learning  Plans  (PLP) 


Subordinates’  goals  reflect  those  assigned 
to  their  supervisors 


Duties  and  tasks  are  clear  to  subordinates 

•  Current  statements  of  duties 

•  Measureable  objectives 

•  Timely  performance  assessment,  linked  to 
objectives 


Managers  and  staff  have  appropriate  training  and  authority  to 
undertake  their  delegations 

•  Records  of  managers’  certifications 

•  Records  of  mandatory  management  training 


A  formal  review  process  tracks  progress,  delegations  and 
performance 

•  Performance  appraisal  statistics 

Achievement  of  subordinates’  goals  supports  management 
goals  and  the  organizational  vision 

•  Managers’  survey 


DND.  Organization  and  Accountability:  Guidance  for  Members  of  the  Canadian  Forces  and  Employees  of  National 
Defence.  Second  Edition,  Sept  1999,  Ch  IV,  2  URL:  http://www.forces.gc.ca/admpol/Organization-e.html. 
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3.5  Evidence  and  Instruments 


Table  14  summarizes  evidence  or  instruments  that  can  be  used  to  support  the  proposed  DRDC 
MAP.  Not  all  instruments  are  necessary,  since  many  hold  the  same  information.  This 
redundancy  is  fortunate  since  some  instruments  may  be  more  convenient  than  others  for 
supporting  particular  elements  of  the  DRDC  MAP.  The  large  number  of  data  sources  also  serves 
to  reduce  the  need  for  managers’  surveys  to  the  absolute  minimum  required,  thus  reducing  risk  of 
survey  fatigue. 
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Table  14:  Summary  of  Evidence  and  Instruments  for  the  Proposed  DRDC  MAE 


Evidence  or  Instrument 

Defence  R&D  Canada 
Agency  Values 

Governance  and 
Strategic  Direction 

Results  and 

Performance 

Learning,  Innovation 
and  change 

Policy  and  programs 

People 

Service 

Risk  Management 

Stewardship 

Accountability 

Alternative  Despute  Resolution  Statistics 

X 

ATI  Program  Performance 

X 

Audit  and  Program  Evaluation  Reports 

X 

X 

X 

X 

CANDID 

X 

X 

Classification  and  other  grievance  statistics 

X 

Client  /Partner  Satisfaction  Surveys 

X 

X 

CPME 

X 

X 

X 

X 

X 

Development  Program  Participation 

X 

Defence  Ethics  Survey^” 

X 

X 

DRDC  Annual  Report 

X 

X 

X 

DRDC  Business  Plan 

X 

X 

X 

DRDC  Human  Resources  Plan 

X 

DRDC  Inventory  Records  for  Equipment 

X 

DRDC  Managers’  Survey 

X 

X 

X 

X 

X 

X 

X 

DRDC  PAA 

X 

DRDC  Partner  Group  Agreements 

X 

X 

DRDC  Rsik  Profile,  Policies  and  SOPs 

X 

EAP  Activity  Reports 

X 

Environmental  Assessment  Performance 

X 

EXPEDITION  XX  ‘Dashboard’ 

X 

X 

FMAS 

X 

X 

Inventory  of  Scientific  Equipment 

X 

X 

LMRC  and  LMCC  Records 

X 

Peer  Reviews  of  S&T  Programs 

X 

X 

PeopleSoft® 

X 

X 

X 

X 

X 

PMA  Statistics 

X 

X 

Public  Service  Employee  Survey 

X 

X 

X 

Recognition  and  Reward  Program 

X 

The  Defence  Ethics  Survey  would  be  of  considerable  value,  subject  only  to  its  availability  via  DRENet. 
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4  Application  of  the  DRDC  MAF 


This  section  describes  the  application  of  the  proposed  DRDC  MAF.  It  begins  with  a  description 
and  analysis  of  pilot  studies  conducted  during  EXPEDITION  09  and  then  proposes  the 
framework  for  a  steady-state  operational  MAF.  The  findings  in  Section  4.1  are  a  summary  of 
those  reported  in  the  companion  cycle  2  report"^,  which  provides  details  of  responses  to 
individual  questions  in  the  surveys  and  organizes  findings  at  the  individual  Measure  level,  rather 
than  at  the  element  level. 


4.1  Pilot  Studies 

Three  pilot  studies  were  conducted  as  part  of  the  cyclic  development  of  the  DRDC  MAF.  All 
three  pilots  employed  surveys  of  DRDC  Managers.  The  first  pilot  study  examined  the 
Stewardship  element,  and  demonstrated  the  feasibility  of  the  survey  approach.  The  second  pilot 
study  examined  the  Accountability  and  Risk  Management  elements.  The  third  pilot  examined 
Policy  and  Programs,  Service  and  People,  and  revisited  Stewardship  to  test  an  annual  cycle.  The 
following  discussion  will  be  organized  on  the  basis  of  the  DRDC  MAF  structure,  rather  than  by 
chronology  of  the  pilots. 

The  pilot  studies  addressed  the  core  elements  of  the  DRDC  MAF  primarily  through  surveys.  The 
surveys  are  described  in  some  detail  in  O’Blenis  and  Nethercote  and  Baroni  and  Nethercote,  and 
so  only  the  findings  of  the  pilot  studies  are  described  here.  The  pilot  study  surveys  demonstrate 
the  feasibility  of  surveys  for  the  DRDC  MAF,  but  also  the  weaknesses  of  surveys.  Table  15 
summarizes  returns  from  the  three  pilot  surveys,  and  the  decline  in  response  rate  {survey  fatigue) 
is  evident.  Clearly,  surveys  are  useful,  but  must  be  employed  effectively,  but  sparingly  and  only 
where  other  data  sources  are  not  available.  This  conclusion  is  supported  by  responses  to  open- 
ended  questions  in  the  third  pilot  survey,  where  complaints  of  survey  fatigue  were  made.  After- 
the-fact  conversations  with  DRDC  managers  indicate  another  issue,  that  the  two-week  availability 
for  the  later  surveys  was  too  short. 

Survey  returns  predominantly  reflect  Eevel  3  managers.  The  numbers  of  available  Eevel  2  and 
Level  4  managers  are  much  lower,  to  the  degree  that  these  two  groups  could  not  be  sorted  by 
centre  without  risk  of  identifying  individuals.  O’Blenis  and  Nethercote  and  Baroni  and 
Nethercote  address  this  in  more  detail. 

The  four  pillar  elements  (Defence  R&D  Canada  Agency  Values;  Governance  and  Strategic 
Direction;  Results  and  Performance;  and.  Learning,  Innovation  and  Change  Management)  were 
not  addressed  explicitly  in  the  pilot  studies,  but  many  of  the  measures  and  instruments  applicable 
to  the  core  elements  will  also  support  the  pillar  elements.  This  will  be  demonstrated  in  Section 
4.2.  Findings  of  the  pilot  studies  for  the  core  elements  follow  in  Sections  4.1.1  through  4.1.6. 


See  Baroni  and  Nethercote,  DRDC  Atlantic  CR2009-135  Sections  7  and  8  for  more  details. 
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Table  15:  DRDC  MAF  Pilot  Survey  Participation 


Population 

(Address 

List"®) 

Valid 

Addresses"^ 

Valid 

Responses 

Response 
from  Valid 
Addresses, 
% 

First  Pilot  (open  3 
weeks:  Summer  2008) 

97 

73 

45 

62 

Second  Pilot  (open  2 
weeks,  February 

2009) 

141 

141 

75 

53 

Third  Pilot  (open  2 
weeks,  June  2009) 

149 

147 

70 

48 

4.1.1  Policy  and  Programs 

According  to  survey  responses,  the  strategic  goals  of  Canada’s  national  S&T  strategy"*  appear  to 
be  adequately  represented  in  DRDC’s  programs,  with  most  activities  addressing  multiple  national 
goals.  The  workforce  development  goal  of  the  national  strategy  is  one  that  is  not  as  well 
represented,  largely  because  DRDC’s  commercialization  and  mission  directed  R&D  activities 
require  a  level  of  expertise  which  is  not  conducive  to  many  conventional  workforce  development 
goals. 

Development  of  policy  capacity,  which  for  DRDC  includes  S&T  capacity,  is  often  problematic  in 
DRDC.  Many  managers  feel  that  key  positions  are  not  filled  quickly  enough,  though  once  they 
are  filled,  the  individual  performs  effectively  in  a  reasonable  timeframe.  Additionally,  there  is  no 
relationship  between  the  identification  of  high  performing  employees  and  how  quickly  a  key 
position  is  filled,  nor  how  effectively  a  new  employee  in  a  key  position  performs.  This  may  be  a 
reflection  of  lack  of  succession  planning.  Generally,  despite  a  lack  of  a  succession  plan  in  the 
organization,  managers  are  identifying  high  potential  employees  in  their  units  and  seem  to  feel 
that  there  are  employees  ready  to  compete  for  key  positions."^ 

Defence  R&D  Canada’s  mentoring  program  offers  one  component  of  a  successful  succession 
plan.  Most  participants  in  the  DRDC  mentoring  program  have  had  positive  experiences,  and 
would  participate  in  the  program  in  the  future.  Non-participants  appear  to  have  limited 
knowledge  about  the  program,  and  little  desire  to  participate. 


For  the  first  pilot,  the  address  list  was  the  manager  attendee  list  for  the  2008  DRDC  Managers’  Workshop.  For  the 
second  and  third  pilots,  the  address  list  was  those  eligible  for  the  2009  DRDC  Managers’  Workshop,  a  considerably 
larger  number.  Attendance  at  the  DRDC  Managers’  Workshop  is  normally  limited  to  about  100  managers,  thus  giving 
rise  to  the  moniker,  “The  One  Flundred,”  used  to  describe  DRDC’s  management  cadre. 

The  number  of  valid  addresses  reflects  the  number  of  survey  invitations  that  did  not  ‘bounce  back’  as  invalid.  For 
the  first  pilot,  invalid  addresses  also  include  those  which  returned  answerback  messages  saying  that  the  addressees 
would  be  absent  on  annual  leave  for  the  duration  of  the  survey.  The  first  pilot  was  executed  during  prime  holiday 
season:  the  last  two  weeks  of  July  and  the  first  week  of  August.  Pilots  two  and  three  were  conducted  during  winter  and 
spring,  and  no  answerbacks  were  received  indicating  that  addressees  were  on  vacation  for  the  duration  of  the  survey. 

*  **  Industry  Canada,  Mobilizing  Science  and  Technology  to  Canada ’s  Advantage,  Ottawa:  2007. 

***  In  the  experience  of  one  of  the  authors  it  is  entirely  another  matter  whether  employees  ready  to  compete,  will 
compete.  The  attractiveness  of  management  is  the  topic  of  the  Management  Leadership  Development  task  in  WBE  1 .4 
of  EXPEDITION  09. 
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As  may  be  expected  in  an  S&T  based  environment,  most  managers  report  that  keeping  up  with 
new  ideas,  methods  and  technology  is  very  important  to  their  job  performance;  however,  there  are 
mixed  reports  as  to  the  satisfaction  with  resources  available  to  do  so. 

4.1.2  People 

Although  the  Public  Service  Employee  Survey'^®  is  the  preferred  survey  instrument  for  the  People 
element  of  the  DRDC  MAE,  several  relevant  questions  were  asked  of  Eevel  2  managers  during 
the  third  pilot  survey.  Neither  the  population  nor  the  response  rate  for  level  2  managers  support 
confidence  in  the  results;  nonetheless,  only  one -third  of  the  six  level  2  respondents  felt  that 
employment  equity  groups  were  adequately  represented  in  feeder  groups  for  key  positions.  Half 
of  Level  2  respondents  reported  offering  developmental  opportunities  specifically  for 
employment  equity  groups. 

4.1.3  Service 

The  EXPEDITION  series  of  organizational  change  initiatives  are  themselves  indicative  of  an 
organization  whose  leadership  is  committed  to  transformation  and  service  improvement. 

Managers  appear  to  be  unaware  of  the  results  of  the  most  recent  client  satisfaction  surveys, 
although  this  may  be  in  part  due  to  suspension  of  such  activities  during  stand-up  of  Defence  S&T 
Enterprise  processes.  Overview  Group  or  Thrust  Advisory  Group  records  can  supplant  missing 
client  surveys,  but  cannot  replace  the  candor  of  responses  available  from  anonymous  client 
surveys.  In  the  absence  of  such  information,  managers’  own  perception  of  service  quality  may  be 
misaligned  with  client  or  partners’  perceptions,  to  the  detriment  of  client/partner  satisfaction. 

Survey  data  point  to  better  quality  of  internal  collaboration  than  external  collaboration  (industry 
and  academe).  Survey  results  also  indicate  that  higher  quality  collaborative  relationships  lead  to 
increases  in  goal  achievement. 

4.1.4  Risk  Management 

Risk  management  has  been  problematic  for  DND  in  the  TB  MAE,  and  must  necessarily  be  so  for 
DRDC  in  its  own  MAE.  A  departmental  Corporate  Risk  Profile  (CRP)  is  the  cornerstone  of  the 
TBS  Risk  management  element.  The  Department  of  National  Defence  has  not  had  a  CRP, 
although  one  is  reportedly  close  to  approval  and  release. 

There  appears  to  be  no  comprehensive  DRDC  risk  management  profile,  which  should  be  no 
surprise  in  absence  of  a  DND  CRP.  There  is  neither  a  process  in  place  to  identify  risk 
management  stakeholders,  nor  a  communication  plan  to  communicate  with  stakeholders. 
Stakeholder  identification  appears  to  be  based  on  corporate  memory  of  past  consultations;  this 
knowledge  should  be  captured  in  a  data  system,  and  an  approach,  such  as  an  environmental  scan, 
should  be  developed  to  identify  stakeholders  who  may  not  have  been  considered  in  the  past. 


™  See  “2008  Public  Service  Employee  Survey,”  Available  at:  http://www.tbs-sct.gc.ca/svdg/pses-eng.asp. 
™  Karim  Dahel,  Private  communication. 

An  examination  of  Descartes  revealed  no  policy  statement  or  SOP. 
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Risk  management  is  often  weak  among  the  DRDC  management  cadre,  although  evidence  of 
expertise  exists.  It  is  commonly  held  that  risk  management  is  the  purview  of  project  managers, 
rather  than  a  pervasive  responsibility  in  the  organization.  This  project  manager-centric  view  is 
consistent  with  concentration  of  risk  management  activities  in  the  Technology  Demonstration 
Program  (TDP).  There  is  a  need  for  staff  development  with  respect  to  risk  management,  and  for 
inculcation  of  risk  management  practices  outside  the  TDP  program. 

Level  3  Managers  do  not  appear  to  consistently  have  confidence  that  their  subordinates 
understand  their  roles,  responsibilities,  or  limitations  on  authority  when  managing  risks.  In  each 
case,  over  half  of  respondents  either  have  inconsistent  or  no  confidence  that  their  subordinates 
understand  these  risk  management  duties.  Statistical  testing  supports  the  assumption  that 
managers  who  make  their  risk  management  priorities  and  tolerances  clear  to  subordinates  are 
more  likely  to  have  confidence  in  their  subordinates’  understanding  of  their  roles  in  managing 
risks. 

Managers  who  report  that  project  plans  consistently  identify  the  likelihood  and  impact  of  risks 
appear  more  likely  to  revisit  risk  assessments  during  the  delivery  of  the  project.  This  suggests 
that  those  who  explicitly  identify  project  risks  through  a  likelihood/  impact  matrix  are  more  likely 
to  revisit  risk  assessments  during  the  delivery  of  that  project.  Mangers  who  report  ‘always’  or 
‘often’  when  comparing  progress  to  established  project  milestones  also  appear  somewhat  more 
likely  to  report  ‘always’  or  ‘often’  when  identifying  the  likelihood  and  impact  of  project  risks. 

These  findings  reflect  the  conventional  vision  of  risk  as  failure,  as  embodied  in  formal, 
conventional  risk  management  practices  promoted  by  the  TBS.  The  pilot  study  did  not  determine 
the  level  of  failure  in  DRDC  activities,  to  allow  the  significance  of  the  risk  management  findings 
to  be  truly  assessed.  The  risk  as  opportunity  concept  was  not  addressed  in  the  pilot  survey,  so 
that  DRDC’s  fundamental  corporate  role  as  risk  reducer  in  the  international  defence  and  security 
context  was  not  addressed.  This  aspect  of  risk  management  could  be  brought  into  play  in  a 
DRDC  Corporate  Risk  Profile.  Alternatively,  the  risk  as  opportunity  concept  could  be  addressed 
through  the  Policy  and  Programs  element,  although  doing  so  would  obscure  rather  than  reveal  the 
tension  between  risk  as  failure  and  risk  as  opportunity.  Annex  A  to  this  report  explores  the 
application  of  culture  theory  to  risk  management  as  a  means  for  DRDC  to  recognize  and  exploit 
risk  as  opportunity  concepts. 

4.1.5  Stewardship 

Stewardship  was  examined  in  two  pilots,  the  first  and  the  third.  This  allowed  demonstration  of  an 
annual  cycle  of  a  MAP.  The  third  pilot  survey  (S3)  revisited  areas  of  interest  from  the  first  pilot 
survey  (SI).  Areas  which  garnered  incomplete  or  unsatisfactory  responses  in  the  first  pilot  were 
of  particular  interest.  Overall,  there  was  little  change  in  responses  from  SI  to  S3,  which  is  not 
unexpected  since  there  was  no  organizational  response  to  the  first  pilot  survey,  as  there  would  be 
for  an  operational  MAP. 

Budgeting.  The  first  pilot  survey  indicated  that  over  half  of  respondents  with  budgeting 
responsibilities  had  budget  tolerances  set  by  their  superior,  and  that  of  those,  all  but  one 
individual  successfully  managed  their  budget  within  that  tolerance.  The  third  pilot  survey 
investigated  the  significance  of  setting  budget  tolerances,  and  found  that  there  appears  to  be 
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benefits  to  the  organization.  By  setting  budget  tolerances  and  supporting  employees’  efforts  to 
work  within  that  tolerance,  managers  are  encouraging  delegation  and  a  culture  of  accountability. 

Capital  Planning.  As  in  the  first  pilot  survey,  respondents  in  S3  indicate  that  too  few  units  have 
long  term  capital  plans,  which  are  necessary  for  building  and  maintaining  capacity.  Those  which 
do  have  capital  investment  plans  based  on  long  term  needs  appear  to  be  working  towards  future 
capacity  needs  incrementally.  In  S 1 ,  response  trends  suggest  that  many  managers  view  capital 
planning  as  a  frustrating  exercise,  particularly  when  Vote  5  funding  is  generally  in  short  supply. 

Procurement.  The  first  pilot  survey  found  that  there  was  substantial  dissatisfaction  with  the 
procurement  processes,  while  S3  also  determined  that  there  appears  to  be  too  few  procurement 
specialists  for  the  level  of  procurement  activity  in  many  centres.  The  first  pilot  survey  found  that 
the  majority  of  respondents  generally  or  always  had  sound  working  relationships  with 
procurement  functional  authorities;  survey  3  investigated  further  and  found  that,  overall,  the 
procurement  process  and  the  relationships  with  procurement  specialists  appear  to  be  better  at 
traditional  research  centres than  at  NDHQ  centres.  Traditional  research  centres  were  also 
clearly  better  than  NDHQ-based  centres  at  making  progress  of  procurement  files  visible  to 
clients,  and  responses  indicate  that  of  those,  DRDC’s  Atlantic  and  Valcartier  may  be  a  source  of 
best  practices. 

Projects.  There  continues  to  be  a  tendency  to  attribute  project  failures  primarily  to  schedule  and 
resource  issues.  In  the  third  pilot  survey,  as  in  the  first,  it  was  noted  that  documentation  for 
projects  managed  within  a  manager’s  unit  did  not  include  a  responsibility  matrix  or  some  other 
formal  assignment  of  responsibilities. 

Records.  Both  pilot  surveys  found  an  overall  good  level  of  accountability  for  inventory  items. 
Both  identified  areas  of  weakness  in  record  keeping  in  general  and  in  securing  HR  documents  in 
particular.  In  both  surveys,  records  were  reported  as  not  being  sufficiently  complete  for  the 
purposes  of  an  audit,  and  in  both  surveys,  personnel  fdes  are  accessible  to  non-HR  staff'^^. 
Survey  3  followed  up  on  the  security  and  tracking  of  classified  documentation,  and  found  some 
improvement  over  Survey  1 ,  which  may  be  due  to  clarification  of  the  question;  however,  there  is 
still  room  for  improvement  in  the  tracking  of  classified  correspondence. 

4.1.6  Accountability 

There  is  a  clear  recognition  of  the  principles  of  accountability  by  DRDC  managers,  but  execution 
of  accountability  relationships  is  often  inadequate.  Objective  setting  is  generally  based  upon  the 
requirements  of  superiors  and  corporate  objectives,  but  those  objectives  frequently  lack 
measurable  performance  standards,  so  that  it  is  difficult  to  hold  subordinates  to  account.  Level  3 


Atlantic,  Valcartier,  Ottawa,  Toronto,  and  Suffield. 

“DRDC  Atlantic  and  DRDC  Valcartier  are  the  only  centres  using  ORACLE-based  applications  to  support  local 
administrative  functions,  such  as  procurement.”  Private  communication,  D.C.  Oxford. 

Note  of  caution:  the  surveys  asked  managers  “if  access  to  HR  files  was  restricted  to  HR  personnel  only”.  Did 
managers  respond  based  on  personal  experience  of  access  refused  or  granted,  did  they  have  incorrect  understandings  of 
policy,  or  did  their  interpretation  of  ‘access  to  files’  differ  from  that  of  the  survey  designers?  An  example  of 
misconception  became  apparent  during  focus  group-based  testing  of  S3,  when  a  new  manager  was  surprised  to  learn 
that  personnel  files  would  not  be  available  to  him  directly;  instead,  he  would  have  to  obtain  information  from  personnel 
files  through  HR  staff.  An  example  of  differing  interpretations  would  be  that  obtaining  information  via  HR  personnel 
was  tantamount  to  ‘access’,  which  was  not  the  survey  designers’  understanding. 
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managers  who  do  set  measurable  performance  standards  for  their  subordinates  also  report 
receiving  measurable  performance  standards  from  their  superiors.  This  reinforces  the  important 
role  of  Level  2  managers  in  establishing  effective  accountability.  Level  2  managers  are  role 
models  for  their  subordinates. 

Only  half  of  Level  3  managers  set  performance  objectives  for  their  subordinates  using  milestones 
in  CPME.  Thus,  a  ‘quick  hit’  opportunity  for  greater  use  of  measurable  performance  standards 
exists:  Level  3  managers  should  build  stronger  links  between  individual  performance 
management  (through  goal  setting,  PERs  and  PRRs)  and  program  or  project  management 
(through  CPME  milestones). 

On  a  positive  note,  Eevel  3  managers  reported  various  effective  strategies  to  address  performance 
deficiencies,  ranging  from  consultation,  through  developmental  training,  to  engagement  of  HR 
staff  Ineffective  strategies  were  few. 

Eevel  3  managers  appear  to  be  better  at  distinguishing  between  duties  and  tasks  for  their 
subordinates,  than  their  superior  Eevel  2  managers  are  at  distinguishing  between  duties  and  task 
for  Eevel  3s. Those  managers  who  do  make  a  distinction  between  duties  and  tasks  appear  to 
have  more  effective  working  relationships  with  their  subordinates. 

4.2  Steady-state  Application 

An  options  analysis  was  used  to  develop  a  proposed  implementation  strategy  for  a  DRDC  MAE. 
Options  were:  (1)  linking  a  DRDC  MAE  cycle  to  the  TB  MAE  cycle;  (2)  linking  a  DRDC  MAE 
cycle  to  the  Executive  PMA  cycle;  and  (3)  linking  a  DRDC  MAE  cycle  to  the  DRDC  business 
cycle.  Option  2  was  easily  dismissed.  Both  options  1  and  3  were  feasible,  but  since  ADM(S&T) 
is  not  a  Departmental  OPl  for  any  of  the  TB  MAE  elements,  there  is  potentially  greater  benefit  in 
selecting  option  3.  In  any  event,  the  TB  MAE  cycle  is  integrated  with  the  Government  of  Canada 
(GoC)  business  cycle;  therefore,  a  DRDC  MAE  integrated  with  the  DRDC  business  cycle  (and  so 
the  DND  and  GoC  business  cycles)  will  necessarily  be  in  alignment  with  the  TB  MAE  cycle. 
Thus  a  blend  of  option  1  and  3  was  developed,  as  shown  in  Eigure  7,  overleaf 

The  following  discussion  describes  the  steps  and  associated  resources  in  the  proposed  DRDC 
MAE  cycle.  The  discussion  identifies  proposed  OPls  and  assumes  a  steady-state  situation. 

4.2.1  Beginning  at  the  End 

The  beginnings  of  a  DRDC  MAE  cycle  lie  in  the  end  of  the  last,  where  the  output  of  the  MAE 
processes  identify  strengths  and  weaknesses  in  DRDC  management  processes.  Strengths  offer 
opportunity  for  commendation  whereas  weaknesses  offer  opportunity  for  organizational 
development  or  process  improvement.  Thus,  the  end  of  the  previous  cycle  presents  the 
opportunity  for  RDEC  to  identify  priorities  for  the  next  MAE  cycle. 


For  example,  “Either  I  repeat  what  my  expectations  are  or  I  change  my  expectations.”  (!) 

The  TB  MAF  speaks  of  accountabilities  for  process  (here,  ‘duties’)  and  results  (here,  ‘tasks’). 
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Figure  5:  Annual  Cycle  for  the  proposed  DRDC  MAF. 


This  consideration  and  direction  by  RDEC  is  part  of  DRDC’s  normal  executive  overhead,  so  no 
MAF-related  resource  estimate  is  offered.  Figure  7  shows  this  RDEC  activity  happening  in 
December,  but  it  might  equally  occur  at  any  point  during  the  Autumn  timeframe. 


4.2.2  Planning  the  DRDC  MAF  Cycle 

The  Director,  R&D  Program  Oversight  (DSTEA)  is  the  proposed  OPl  for  the  DRDC  MAF  and 
will  be  responsible  for  planning  the  annual  DRDC  MAF  cycle  based  upon  the  priorities  identified 
by  RDEC  at  the  completion  of  the  previous  MAF  cycle.  The  planning  stage  would  evolve  in  two 
stages: 

•  The  first  would  examine  RDEC  priorities  and  develop  a  MAF  strategy,  including 
elements  of  interest  and  appropriate  measures  and  data  sources;  and, 

•  The  second  would  identify  required  data  not  available  corporately  and  use  these 
requirements  to  develop  themes  and  core  questions  for  a  survey  of  managers. 

It  is  estimated  that  these  activities  would  require  four  (4)  person-weeks  of  DSTEA  staff  effort 
within  the  February  timeframe. 
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4.2.3  Collecting  Corporate  Information 

Based  on  the  results  of  the  planning  exercise,  DSTEA  will  request  information  of  interest  that  is 
available  from  corporate  sources,  from  DGRDCS  and  DGSTO  staffs.  Such  information  might 
include,  for  example,  milestone  performance  from  CPME,  financial  performance  from  FMAS, 
training  statistics  from  PeopleSoft®,  or  ethical  performance  from  the  Defence  Ethics  or  Public 
Service  Employee  Surveys. 

It  is  estimated  that  these  activities  would  require  five  (5)  person-weeks  of  staff  effort  from 
DGRDCS  and  DGSTO  staff,  combined.  This  effort  would  be  spread  over  the  March  through 
May  timeframe  to  ensure  that  end-of-year  data  were  available  if  required. 

4.2.4  Develop  and  Test  Survey 

The  Director  General,  Military  Personnel  Research  and  Analysis  (DGMPRA)  is  the  proposed  OPl 
for  development  and  testing  of  the  on-line  survey  instrument  to  be  used  to  address  those  issues 
not  covered  by  corporate  data  sets.  Such  issues  will  ordinarily  be  related  to  direct  determination 
of  managers’  perceptions  or  performance  in  practice,  rather  than  relying  upon  assumed 
compliance  with  policies  or  SOPs  as  proxies  for  managerial  performance. 

The  survey  instrument  will  require  translation  and  will  require  Social  Science  Research  Review 
Board  (SSRRB)  waiver  or  approval.  For  the  third  pilot,  the  SSRRB  was  asked  to  consider 
whether  the  DRDC  MAE  surveys  actually  represented  research  on  humans,  or  whether  they 
represented  “quality  assurance  [or]  performances  reviews”  which  were  exclusions  from  the 
requirements  for  ethical  review  in  the  Tri-Council  Statement.  The  SSRRB  addressed  the  question 
as  follows: 

Board  members  considered  it  to  be  more  in  line  with  a  program  evaluation  than  "social 
science"  research  because  the  questions  are  not  of  a  personal  nature,  they  are  specifically 
geared  towards  receiving  feedback  and  evaluation  and  that  SSRRB  review  was  not 
necessarily  required.  Generally  this  type  of  research  does  not  get  sent  to  the  SSRRB  for 
review  -  it  is  just  issued  a  coordination  number.  ™ 

Thus,  it  would  appear  that  operational  MAE  surveys  would  be  exempt  from  SSRRB  review, 
although  a  release  approval  (‘coordination  number’)  by  DGMPRA  would  still  be  required.  This 
would  not  be  an  issue  of  any  significance  given  that  DGMPRA,  the  OPl  for  the  DND  SSRRB,  is 
proposed  as  the  survey  instrument  OPl. 

It  is  estimated  that  these  activities  would  require  five  (5)  person-weeks  of  effort,  exclusive  of 
translation,  in  the  April  timeframe.  It  is  recognised,  based  on  the  experience  of  the  pilot  studies, 
that  translation  services  would  be  a  critical  path  item  in  the  development  of  any  survey. 


128  Policy  Statement:  Ethical  Conduct  for  Research  Involving  Humans,”  Public  Works  and  Government 

Services  Canada,  Ottawa:  2003,  Article  1 . 1  (d),  page  1.1. 

™  Email  Aker  -  Nethercote,  “RE:  For  SSRRB  Review:  Third  MAF  Survey,”  Mon  30/04/2009  3:06PM. 
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4.2.5  Management  Survey  Active 

Since  DGMPRA  is  the  OPI  for  survey  development,  it  is  natural  that  it  be  the  OPI  for  survey 
delivery.  The  on-line  survey  tool  now  used  by  DGMPRA,  E-Listen,  is  currently  limited  to 
DWAN/DIN.  A  DRENet-compatible  survey  instrument  will  be  required  for  DRDC  MAE 
surveys  and  would  also  allow  the  2010  Defence  Ethics  Survey,  or  a  subset  thereof,  to  be 
administered  to  DRDC  staff  Resource  estimates  for  implementation  of  a  DRENet  survey 
capability  by  DGMPRA  have  not  been  made.  The  DRDC  MAE  pilot  studies  used  a  survey  utility 
on  DRDC  Atlantic’s  ORACLE®  server.  Dalhousie  University  (the  DRDC  MAE  contractor)  uses 
the  OPINIO®'^'  application  as  its  corporate  web-based  survey  tool,  but  an  enterprise  license  for 
OPINIO  is  required  to  avoid  privacy  and  security  issues  related  to  the  US  PATRIOT  Act.'^^ 

The  on-line  survey  would  be  available  to  the  DRDC  management  cadre  for  four  weeks.  It  is 
recommended  that  survey  participation  be  made  a  mandatory  activity,  rather  than  a  recommended 
activity.  Many  survey  tools,  like  OPINIO®,  allow  invitees’  survey  participation  to  be  tracked 
without  comprimising  anonymity  of  responses. 

The  survey  would  be  active  in  the  May  timeframe,  requiring  about  one-half  (Vi)  person-week  of 
activity  for  the  survey  manager,  and  a  total  of  about  four  (4)  person-weeks  of  activity  for 
managers’  responses  to  the  survey.  Recommended  rather  than  mandatory  participation  would 
likely  reduce  participation  and  resource  requirements  by  half. 

4.2.6  Survey  and  Data  Analysis 

DSTEA  would  be  the  OPI  for  survey  and  data  analysis,  supported  by  DGMPRA  staff,  who  would 
deliver  analysis  of  survey  results,  including  cross-tabulations.  DSTEA  staff  would  aggregate  the 
various  data  sets,  including  survey  results,  and  analyze  them  to  support  assessment  of  those 
DRDC  MAE  elements  of  interest  to  RDEC. 

It  is  estimated  that  these  activities  would  require  three  (3)  person-weeks  of  DGMPRA  staff  effort 
and  six  (6)  person-weeks  of  DSTEA  staff  effort  in  the  June  timeframe. 

4.2.7  Reporting 

DSTEA  would  be  the  OPI  for  reporting.  Initial  effort  in  the  early  July  timeframe  would  be 
directed  towards  key  results  reporting  for  the  RDEC  Retreat.  Complete  reporting  would  follow. 

It  is  estimated  that  four  (4)  person-weeks  of  DSTEA  staff  effort  would  be  required  for  reporting 
over  the  July  to  September  timeframe. 


Email  Bowser-Nethercote,  “Re:  Upcoming  Survey  for  EXPEDITION  09,”  Mon  06/04/2009  5:39PM. 

See  http://www.obiectpIanet.com/opinio/. 

™  Dalhousie  University’s  enterprise  license  for  OPINIO®  provides  for  an  executable  source  on  a  Dalhousie  University 
server,  thus  protecting  Dalhousie  University  data  against  the  PATRIOT  Act. 
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4.2.8  Support  to  TB  MAF 

Support  to  the  TB  MAF  might  be  requirred  on  an  ongoing  basis,  but  should  be  regarded  as  a 
corporate  overhead,  rather  than  a  charge  to  the  DRDC  MAF. 
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5  Recommendations  and  Conclusions 


This  section  of  the  report  provides  recommendations  and  conclusions  concerning  two  themes. 

The  first  theme  is  the  DRDC  MAP  and  particularly  its  adoption  operationally.  The  second  theme 
is  organizational  performance  of  DRDC,  as  determined  during  the  cyclic  development  of  the 
DRDC  MAP.  The  recommendations  in  this  section  are  a  summary  of  those  reported  in  the 
companion  cycle  2  report, which  provides  considerably  more  detail.  The  section  closes  with  a 
reflection  upon  DRDC  as  an  organization  that  represents  risk  management  in  its  own  right. 

The  research  undertaken  in  EXPEDITION  09  leads  to  the  following  recommendations  concerning 
the  proposed  DRDC  MAP: 

•  It  is  recommended  that  DRDC  adopt  the  proposed  DRDC  MAP,  as  described  herein.  There 
are,  among  others,  a  number  of  observations  of  note: 

o  The  pilot  studies  support  the  feasibility  and  effectiveness  of  surveys  as 

instruments  for  the  MAP,  but  declining  participation  throughout  the  pilot  survey 
process,  together  with  respondent  comments,  point  to  the  need  to  limit  surveys  to 
a  single  annual  offering,  and  to  restrict  questioning  to  those  data  which  cannot  be 
obtained  elsewhere.  Where  genuinely  unique  surveys  are  the  norm,  it  is 
recommended  that  survey  participation  be  mandatory  for  managers, 
o  DGMPRA  has  an  in-house  capability  for  the  cost-effective  execution  of  surveys 
through  the  E-EISTEN  tool.  That  survey  capability  is  unfortunately  limited  to 
the  DWAN/DfN.  Defence  R&D  Canada  should  implement  a  DRENet- 
compatible  survey  capability  at  DGMPRA,  probably  through  E-EISTEN  so  that 
DGMPRA  could  execute  surveys  to  all  DRDC  managers  and  staff,  most  of  whom 
do  not  have  DWAN/DIN  access.  This  would  support  the  DRDC  MAP  and 
permit  the  inclusion  of  DRDC  staff  in  the  next  Defence  Ethics  Survey  in  2010. 
o  The  SSRRB  approval  process  for  surveys  requires  submission  of  data  and  reports 
to  DGMPRA.  This  archival  practice  points  to  the  value  of  information  and 
knowledge  management  around  the  DRDC  MAP  process.  Both  DGMPRA  and 
DSTEA  should  be  engaged  in  information  and  knowledge  management  (IKM) 
for  the  DRDC  MAP.  The  DRDC  R&D  Executive  Committee  should  identify  the 
most  appropriate  OPl  for  this  activity. 

o  DRDC  should  exploit  the  availability  of  raw  data  from  the  Public  Service 
Employee  Survey  to  support  the  People  element. 

•  Although  the  proposed  DRDC  MAP  is  itself  a  product  of  EXPEDITION  09,  future  surveys  of 
DRDC  managers  for  the  proposed  DRDC  MAP  would  offer  an  excellent  means  of 
determining  the  organizational  adoption  of  the  outputs  of  the  EXPEDITION  XX  series  of 
organizational  development  projects. 

The  pilot  studies  undertaken  in  EXPEDITION  09  lead  to  the  following  observations  about  the 
performance  of  management  in  DRDC  and  recommendations  for  organizational  development. 


See  Baroni  and  Nethercote,  DRDC  Atlantic  CR2009-135  Section  9  for  more  detailed  recommendations. 
Email  Bowser-Nethercote,  “Re:  Upcoming  Survey  for  EXPEDITION  09,”  Mon  06/04/2009  5:39PM. 
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While  the  tenor  of  many  of  the  observations  is  negative,  the  general  impression  of  DRDC 

management  practices  was  positive. 

•  In  support  of  Policy  and  Programs,  there  is  a  need  for  a  formal,  agency-wide  succession  plan, 
so  that  the  full  career  potential  of  managers  is  realized,  to  the  benefit  of  DRDC’s  policy 
capacity.  Stronger  succession  planning  might  also  contribute  to  increased  representation  of 
employment  equity  groups  in  the  management  cadre.  Exit  and  post-employment  surveys  of 
student  employees  (Co-ops,  Defence  Research  Assistants,  FSWEPs,  etc.)  could  assess 
DRDC’s  attractiveness  as  a  recruiter,  both  of  students,  and  of  employees  at  large. 

•  The  People  element  was  not  a  key  focus  of  the  pilot  studies,  aside  from  observations, 
consistent  with  DRDC’s  Annual  Report, that  employment  equity  groups  were  under¬ 
represented  in  the  management  cadre. 

•  DRDC  should  re -institute  the  practice  of  regular  client  satisfaction  surveys  (presumably  to  be 
known  as  partner  group  surveys,  following  the  Defence  S&T Strategy  vernacular)  to  ensure 
the  availability  of  frank  assessment  of  DRDC  services. 

•  Risk  Management,  as  envisioned  by  Treasury  Board,  is  often  weak  among  the  DRDC 
management  cadre,  although  evidence  of  expertise  exists.  It  is  commonly  held  that  risk 
management  is  the  purview  of  project  managers,  rather  than  a  pervasive  responsibility  in  the 
organization.  This  project  manager-centric  view  is  consistent  with  concentration  of  formal 
risk  management  activities  in  the  TDP  program.  There  is  a  need  for  staff  development  with 
respect  to  risk  management,  and  for  inculcation  (or  recognition  where  appropriate)  of  risk 
management  practices  outside  the  TDP  program.  Care  is  required  to  ensure  that  bureaucratic 
risk  management  practices  are  not  advocated  to  the  detriment  of  risk  as  opportunity,  which  is 
a  central  characteristic  of  S&T  organizations  like  DRDC. 

•  The  quality  of  Stewardship  across  DRDC  was  found  to  be  variable,  but  generally  good. 

Better  capital  planning  would  serve  to  strengthen  DRDC’s  S&T  capacity.  Some  internal 
procurement  processes  require  improvement,  but  fortunately  best  practices  seem  to  be 
available  within  DRDC.  Project  management  practices  appear  to  be  recognized,  but 
improvement  is  required  in  areas  such  as  development  of  responsibility  matrices  outside  the 
TDP  program. 

•  There  is  a  clear  recognition  of  the  principles  of  accountability  by  DRDC  managers,  but 
execution  of  accountability  relationships  is  often  inadequate.  Objective  setting  is  generally 
based  upon  the  requirements  of  superiors  and  corporate  objectives,  but  those  objectives 
frequently  lack  measurable  performance  standards,  so  that  it  is  difficult  to  hold  subordinates 
to  account.  Eevel  3  managers  who  do  set  measurable  performance  standards  for  their 
subordinates  also  report  receiving  measurable  performance  standards  from  their  superiors. 
This  reinforces  the  important  role  of  Level  2  managers  in  establishing  effective 
accountability.  Level  2  managers  are  role  models  for  their  subordinates. 

Risk  Management  in  the  TB  MAE  focuses  upon  identifying,  avoiding,  or  in  the  worst  case 

mitigating  the  effects  of  ‘bad  things’  or  ‘dangers,’  the  ubiquitous  view  of  risk  as  failure.  The 


DRDC  Annual  Report  for  the  year  ending  31  March  2008,  p.  61. 
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DRDC  MAF  introduces  the  concept  of  exploiting  risk  in  its  Expectations  statement,  but  largely  as 
a  result  of  Treasury  Board’s  focus  on  failure,  and  the  need  to  support  the  TB  MAF,  the  Indicators 
and  Measures  for  the  DRDC  MAF  focus  on  the  negative  side  of  risk.  Y et  DRDC  is  an 
organization  that  represents  risk  management  in  its  own  right,  an  organization  whose  mission 
statement  begins: 

DRDC's  mission  is  to  ensure  that  the  Canadian  Forces  are  technologically  prepared  and 

operationally  relevant  ... 

and  whose  mission  statement  includes  a  requirement  to  “anticipat[e]  and  advis[e]  on  future 
science  and  technology  trends,  threats  and  opportunities.”’^^  Defence  R&D  Canada’s  very 
mission  reflects  science  and  technology  as  a  risk  reduction  measure  in  the  international  defence 
and  security  context;  thus,  risk  management  is  ubiquitous  in  DRDC. 

The  results  of  the  DRDC  MAF  survey  of  managers  do  not  reflect  the  ubiquity  of  risk 
management  in  DRDC,  largely  due  to  a  Treasury  Board-based  focus  on  avoidance  of  failure. 

This  focus  may  not  be  warranted,  if  DRDC  does  not  experience  an  unusually  high  number  of 
program  failures.  Information  on  incidence  of  failure  was  not  available  for  the  pilot  studies  and 
so  no  conclusions  can  be  drawn  here,  but  it  is  important  that  DRDC  determine  the  incidence  of 
failure  to  ensure  that  the  operational  application  of  the  DRDC  Risk  Management  element  does 
not  so  formalize  risk  management  that  risk  as  opportunity  is  forgotten,  to  the  detriment  of 
DRDC’s  mission.  Quigley’s  Annex  to  this  report  examines  risk  management  in  the  context  of 
culture  theory  and  offers  developmental  training  recommendations  that  would  support  a  risk 
management  regime  in  DRDC  that  balanced  the  tension  between  failure  and  opportunity. 

Notwithstanding  this  caveat  about  risk  management,  the  proposed  DRDC  MAF  described  herein 
would  meet  the  needs  of  an  operational  DRDC  MAF,  and  the  findings  of  the  pilot  studies  would 
provide  a  sound  baseline  for  the  first  operational  cycle  of  a  DRDC  MAF. 


DRDC  Annual  Report  for  the  year  ending  3 1  March  2008,  p.  3. 
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Annex  A  Using  Cultural  Theory  to  Examine  Risk 

Management  Practices:  A  Commentary  for 
Defence  Research  and  Development  Canada 

by  Kevin  Quigley,  PhD 


A.1  Introduction  and  Key  Observations 

The  overall  aim  of  this  paper  is  to  introduce  a  theoretical  framework  to  examine  and  understand 
risk  management,  and  to  use  this  framework  to  stimulate  thinking  about  risk  management 
challenges  and  potential  solutions  at  Defence  Research  and  Development  Canada  (DRDC).  The 
paper  draws  on  the  risk  literature  from  the  social  sciences  generally  and  from  Cultural  Theory  in 
particular  (Douglas  1982;  1992). 

In  addition  to  this  first  section,  this  paper  has  four  additional  sections. 


•  Section  A.2  summarizes  Cultural  Theory. 

•  Section  A.3  considers  how  the  insights  from  Cultural  Theory  can  help  us  to 
understand  risks  and  risk  governance  at  public  agencies  such  as  DRDC  and  how  one 
might  respond  to  these  risks. 

•  Section  A.4  draws  conclusions  and  proposes  some  relevant  training  for  public 
managers. 

•  Section  A.5  provides  a  reference  list  for  this  appendix,  separate  from  the  bibliography 
for  the  main  body. 


A.1.1  Key  Observations 

Section  A,2,  Cultural  Theory:  Risk,  Blame  and  Good  Governance 

Culture  Theory  suggests  that  a  person’s  sense  of  value  and  risk  is  determined  by  formal  and 
informal  regulation  and  social  integration.  Drawing  on  these  two  variables,  the  Culture  Theory 
typology  describes  four  ‘types’  that  emerge,  each  with  its  preferred  governance  structure,  sense  of 
risk  and  who  or  what  to  blame  when  things  go  wrong. 

A  hierarchist  understands  good  governance  to  mean  a  stable  environment  that  supports 
collective  interest  and  fair  process  through  rule-driven  hierarchical  organizations.  Any  departure 
from  this  rule-bound  hierarchy  represents  risk  for  the  hierarchist.  When  things  go  wrong, 
hierarchists  blame  unclear  and/or  weak  rules,  or  a  lack  of  expertise  within  the  organization. 

An  individualist  understands  good  governance  to  mean  minimal  rules  and  interference  with  free 
market  processes.  Individualists  understand  risk  to  be  government  regulation  of  the  economy  or 
government’s  management  of  public  services.  When  things  go  wrong  they  blame  faulty  incentive 
structures. 


An  egalitarian  understands  good  governance  to  mean  local,  communitarian  and  participative 
organizations.  Egalitarians  understand  risk  to  mean  hierarchies  and  organizations  outside  their 
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system.  When  things  go  wrong,  egalitarians  blame  externals:  ‘management,’  ‘the  executives,’ 

‘the  system,’  ‘Wall  Street.’  They  blame  governments  for  usurping  local  powers  and  processes. 

A  fatalist  understands  good  governance  to  mean  ‘resilience’ — the  capacity  to  bounce  back,  or 
withstand  the  pressures  of  unanticipated  setbacks.  Fatalists  feel  that  having  faith  in  formal 
planning  is  a  risk.  When  things  go  wrong,  they  blame  ‘random  chance’  and  ‘the  cards  that  you 
were  dealt.  ’  Risk  abounds  and  there  is  little  one  can  do  to  mitigate  it. 

Each  type  of  governance  has  strengths  and  weaknesses. 

Cultural  Theory  has  a  potentially  powerful  impact  when  employed  as  a  heuristic  device.  The 
Theory  suggests  people  understand  risk  according  to  their  own  bias,  set  up  governance  structures 
to  protect  their  preferred  systems  and  blame  failings  of  governance  on  the  shortcoming  of  other 
‘types.’  The  Theory  can  help  to  explain  the  recurring  nature  of  certain  risk  debates,  guide  our 
reasoning  about  potential  shortcomings  in  specific  institutional  designs  and  detect  the  types  of 
failures  to  which  these  types  of  design  are  susceptible. 

Section  A,3.  Using  Cultural  Theory  as  a  Guide  to  Examine  Risk  Management  Practices 

Section  111  of  this  paper  is  somewhat  speculative;  it  has  not  been  tested  at  DRDC.  Rather,  the 
discussion  takes  the  insights  of  Cultural  Theory  at  face  value  and  reads  them  into  DRDC,  if  you 
will.  In  many  respects  the  analysis  is  fairly  generic  and  could  be  applied  to  many  public  agencies 
that  perform  functions  similar  to  those  carried  out  at  DRDC. 

For  the  purpose  of  this  discussion,  1  assume  that  DRDC  has  highly  specialized  research  teams  that 
exist  as  units  within  a  bureaucratic  framework. 

Cultural  Theorists  would  argue  we  establish  institutions — such  as  DRDC — ^to  manage  specific 
‘risks,’  which,  according  to  this  theory,  means  ‘bads’  or  ‘dangers.’  If  we  accept  this 
understanding  of  risk  management,  we  see  evidence  of  risk  management  in  each  project  or 
program  that  DRDC  runs,  whether  or  not  it  is  formally  identified  as  ‘risk  management.’ 

Managing  teams  within  bureaucracies  presents  challenges.  While  bureaucracies — which  show 
many  hierarchical  tendencies — expect  those  in  subservient  positions  to  account  formally  for  their 
work  to  their  superiors,  strong  teams — which  show  egalitarian  tendencies — are  generally  not 
outwardly  accountable;  indeed,  the  groups  will  resist  what  they  see  as  ‘outside’  interference.  To 
overcome  some  of  the  challenges  that  this  tension  can  create,  management  might  require  that 
research  teams  devise  or  have  significant  input  into  the  performance  standards  by  which  the 
teams  will  be  held  accountable.  Management  might  also  emphasize  peer  assessments  as  opposed 
to  top-down  methods  of  evaluation. 

There  is  no  perfect  approach  to  managing  risk  and  uncertainty.  There  are  many  risks,  and 
resources  available  to  managing  them  are  limited.  Risk  management  strategies  themselves  can 
generate  risks.  Rather  than  eliminate  risks,  we  often  decrease  certain  risks  by  increasing  other 
risks.  Ultimately  we  seek  to  strike  a  balance:  to  reduce  risks  to  acceptable  levels  of  risk  exposure. 
Public  managers  should  develop  and  practise  sound  judgement  in  how  resources  should  be  spent, 
on  which  risks  and  with  what  goals  in  mind.  Public  managers  should  also  develop  an  awareness 
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of  the  intended  and  unintended  impact  that  their  risk  management  processes  are  having  on  the 
target  risk  as  well  as  related  risks. 

Complex  risks  have  a  multitude  of  potential  causes  and  are  therefore  difficult  to  identify,  quantify 
and  control.  This  level  of  complexity  and  systems  interdependence  can  also  make  it  difficult  to 
hold  people  accountable  for  risk  management.  While  it  is  certainly  important  to  try  to  determine 
the  likelihood  that  failures  will  occur,  public  managers  should  always  maintain  a  level  of 
scepticism  about  their  ability  to  understand — let  alone  control — risk.  Public  managers  should  be 
aware  of  the  different  quantitative  and  qualitative  tools  and  processes  available  for  assessing 
risks.  Once  the  level  of  risk  has  been  estimated  (usually  within  some  range),  risk  management 
tools,  such  as  cost-benefit  analyses,  scenario  planning,  worst-case  scenarios  and  precautionary 
approaches,  can  help  to  develop  appropriate  risk  management  plans.  Different  tools  are 
appropriate  in  different  contexts. 

In  the  case  of  complex  risks,  which  have  a  multitude  of  potential  causes,  public  managers  should 
be  able  to  respond  to  uncertainty,  ambiguity  and  unanticipated  failures.  When  the  complexity  of 
the  risk  makes  it  difficult  to  assign  responsibility  to  individuals,  increasing  the  level  of 
transparency  and  the  frequency  of  reporting  can  be  an  alternative. 

Exploiting  risk  can  also  be  a  springboard  to  innovation.  Innovative  environments  tend  to  be 
deregulated,  competitive,  flat,  and  adaptive  and  offer  private  incentives.  They  also  tend  to 
tolerate  a  level  of  failure  and  conflict.  These  characteristics  are  not  always  achieved  easily  in 
bureaucratic  arrangements.  Public  managers  should  develop  the  organizational  design  skills  to 
motivate  entrepreneurial  behaviour.  Internally,  this  can  include  offering  incentives  for  improved 
performance;  externally,  this  can  include  contracting  with  more  flexible  partners. 

New  and  emerging  social  networking  technologies,  in  particular,  may  present  opportunities  for 
enterprising  public  servants.  Public  mangers  might  facilitate  access  to  new  technologies  through 
strategic  investments  and  deregulation  within  their  organizations. 

A.2  Cultural  Theory:  Risk,  Blame  and  Good  Governance 

The  anthropologist  Mary  Douglas  argues  that  risk — which  she  defines  as  ‘danger’  or  ‘threat’ — 
cannot  be  understood  without  first  considering  values.  A  person  understands  a  source’s  danger  or 
threat  in  relation  to  the  source’s  capacity  to  harm  something  the  person  values.  In  short,  only 
when  we  understand  what  people  value  can  we  understand  what  they  consider  to  be  a  risk. 

Douglas  believes  values,  risk  and  institutions  reinforce  one  another.  What  a  person  thinks 
constitutes  risk  either  to  oneself  or  to  one’s  community'^’  determines  who  or  what  the  person 
blames  when  things  go  wrong.  This  understanding  of  blame  informs  the  person’s  accountability 
system.  The  community’s  institutions,  such  as  the  judiciary,  uphold  this  value  system. 

According  to  Douglas,  a  person’s  attempt  to  change  these  institutions  is  an  effort  to  argue  in 
support  of  different  cultural  values  (1992,  24).  Figure  A1  illustrates  the  concept. 


Douglas  sees  the  micro  and  macro  approach  equally. 
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Figure  A.l:  Values,  Risk  and  Institutions  as  Reinforcing 


Douglas  describes  a  person’s  value  system  in  terms  of  the  grid/group  typology  that  she 
developed.  Grid,  which  we  will  refer  to  here  as  regulation,  measures  the  strength  of  formal  or 
informal  rules  and  social  norms  (1982  191-2).  Group,  which  we  will  call  integration,  measures 
the  extent  to  which  community  constraints  are  imposed  on  an  individual  (1982  191-2).  At  the 
intersection  of  regulation  and  integration,  Douglas  sees  different  ‘types’  of  value  systems 
emerging.  Each  of  these  different  ‘types’  has  different  beliefs  about  what  constitutes  risk.  The 
central  assumption  is  that  there  is  a  relationship  between  modes  of  social  organisation  and 
responses  to  risk,  and  that  risk  and  culture  are  adequately  represented  by  the  dimensions  of  the 
grid/group  scheme.  See  figure  A.2. 


High 

Regulation 


Low 


Fatalist 

Hierarchist 

Individualist 

Egalitarian 

Low  High 

Integration 


Figure  A. 2:  Cultural  Theory  Typology 
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Based  on  this  grid/group  typology,  Hood  (1998)'^*  and  Thompson,  Ellis  and  Wildavsky  (1990) 
examine  the  four  ‘types’  in  more  detail.  They  argue  that  each  type  has  a  distinct  set  of  values, 
understanding  of  risk  and  preferred  governance  arrangement.  Each  governance  arrangement  has 
corresponding  strengths  and  weaknesses,  which  we  will  review  here. 

The  hierarchist  (high  regulation  /  high  integration)  understands  good  governance  to  mean  a 
stable  environment  that  supports  collective  interest  and  fair  process  through  rule-driven 
hierarchical  organizations.  These  institutions  manage  the  society’s  and  the  individual’s  resources 
most  effectively:  the  organization’s  clearly  defined  rules  and  its  capacity  to  recruit  and  assign 
responsibilities  to  subject  matter  experts  enable  fair,  effective  and  efficient  processes  (Hood  1998, 
75).  Any  departure  from  this  rule-bound  hierarchy  represents  risk  for  the  hierarchist.  As  such, 
when  things  go  wrong,  hierarchists  blame  unclear  and/or  weak  rules,  or  a  lack  of  expertise  within 
the  organization.  Their  solution  lies  in  strengthening  and  clarifying  the  reporting  relationships 
and  the  rules  that  govern  the  organization,  and  recruiting  more  experts  (Hood  1998,  53). 

Despite  the  effort  to  make  reporting  relationships  clear,  however,  hierarchical  systems  are 
susceptible  to  people  working  at  cross-purposes.  Although  hierarchies  may  have  performance 
measures,  these  measures  are  designed  by  experts  who  have  their  own  biases.  Similarly,  the 
vastness  of  the  typical  hierarchy  prevents  organizations  and  their  members  from  reacting  quickly, 
absorbs  significant  resources  and  sweeps  indiscretions  ‘under  the  rug.’ 

The  individualist  (low  regulation  /  low  integration)  understands  good  governance  to  mean 
minimal  rules  and  interference  with  free  market  processes.  Individualists  believe  that  people  are 
self-seeking,  rational  and  calculating  opportunists.  Individual  responsibility  rules  supreme  and 
apathy  means  consent  (Thompson,  Ellis  and  Wildavsky  1990,  34  and  65).  Public  service 
provision  aligns  itself  with  private  sector,  market  economy  practices.  Organizations  evolve  from 
the  ground  up  and  focus  primarily  on  the  needs  of  the  ‘customers.’  Rather  than  merely  following 
the  prescribed  rules,  individualists  employ  ‘hands-on’  approaches  to  learning  and  ‘creative’ 
solutions  to  problems  (Hood  1998,  55).  What  hierarchists  understand  to  be  risk — deregulation  of 
the  economy — individualists  see  as  opportunity — the  earning  potential  or  the  market  efficiency 
gains  from  deregulation.  In  contrast,  individualists  understand  risk  to  be  government 
(over-regulation  of  the  economy  by  ‘big’  government)  or  its  management  of  public  services. 

Individualist  practices  have  their  own  pitfalls.  Within  an  organization,  individualist  practices, 
such  as  pay-for-performance,  can  undermine  collective  goals  and  lead  to  a  lack  of  cooperation 
(even  corruption  or  fraud),  as  employees  compete  for  private  gains  instead  of  the  collective  good. 
Moreover,  defining  the  citizen/public  servant  relationship  as  ‘customer/producer,’  as 
individualists  are  wont  to  do,  changes  the  nature  of  the  relationship.  Public  servants  are 
accountable  to  elected  officials,  not  ‘customers.’  Moreover,  treating  citizens  as  customers  can 
threaten  political  equality,  marginalize  the  politically  weak  and  economically  disadvantaged,  and 
undermine  the  government’s  guardianship  responsibilities,  as  the  public  servants  tend  towards 
serving  the  most  present,  vocal  and  powerful  ‘customers’  that  they  have  (Fountain  1999,  2). 

The  egalitarian  (low  regulation  /  high  integration)  understands  good  governance  to  mean  local, 
communitarian  and  participative  organizations.  For  egalitarians,  authority  resides  with  the 


While  Hood  (1998)  draws  from  Douglas’s  work  on  Cultural  Theory,  Hood  uses  different  terms  to  describe  the  four 
‘types’ — ^bossism,  choicism,  groupism,  chancism. 
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collectivity.  Individuals  have  a  responsibility  to  participate  and  the  masses  are  empowered  with 
bottom-up  decision-making.  Moreover,  organizations  are  flat,  or  at  least  there  is  minimal 
difference  between  top  official  and  the  rank  and  file.  Fellow  workers,  not  superiors,  conduct 
performance  appraisals.  And  in  order  to  maximize  transparency,  maximum  information  is 
available  to  workers  and  the  public.  Their  distrust  informs  their  accountability  system:  maximum 
control,  or  face-to-face  accountability.  Egalitarians  understand  risk  to  mean  hierarchies  and 
organizations  outside  their  system.  When  things  go  wrong,  egalitarians  blame  externals: 
‘management,’  ‘the  executives,’  ‘the  system,’  ‘Wall  Street’  (Thompson,  Ellis  and  Wildavsky 
1990).  They  blame  governments  for  usurping  local  powers  and  processes.  Ultimately  they  seek 
solutions  based  on  group  norms  and  reciprocal  processes. 

Egalitarian  systems  strive  for  equality  but  frequently  miss  the  differences.  Egalitarian 
organizations  are  susceptible  to  treating  everyone  in  the  same  manner.  This  one-size-fits-all 
approach  can  often  result  in  splits  or  breakdowns  in  organizations  as  individuals  strive  to  define 
themselves.  While  modem,  decentralized  technologies  may  have  in  some  instances  helped  to 
overcome  some  of  the  challenges  associated  with  distance,  face-to-face  accountability,  maximum 
transparency  and  the  regionalized  interests  of  egalitarians  often  make  it  difficult  and  impractical 
to  operationalize  egalitarian  projects  on  a  large  scale  (Hood  1998,  130).  Moreover,  while 
individuals  working  within  groups  can  motivate  one  another  to  worker  harder,  equally  these 
forms  of  governance  can  result  in  collective  shirking. 

The  fatalist  (high  regulation  /  low  integration)  understands  good  governance  to  mean 
‘resilience’ — the  capacity  to  bounce  back,  or  to  withstand  the  pressures  of  unanticipated  setbacks. 
The  fatalist  rejects  clear-cut  cause  and  effect  relationships.  Risk — like  human  nature — cannot  be 
readily  understood  or  controlled.  In  light  of  this  potentially  chaotic  and  unpredictable  universe, 
fatalists  anticipate  lack  of  cooperation  between  citizens  and  employees  (Thompson,  Ellis  and 
Wildavsky,  1990  35).  Fatalists  manage  by  surprise  techniques,  or  by  circumventing  practised  or 
routine  responses.  When  things  go  wrong,  they  blame  ‘random  chance’  and  ‘the  cards  that  you 
were  dealt.’  Risk  abounds  and  there  is  little  one  can  do  to  mitigate  it. 

The  randomness  that  informs  fatalist  forms  of  governance  undermines  incentives  to  innovate, 
develop  or  compete.  Without  a  clear  cause  and  effect  relationship,  office  holders  are  de¬ 
motivated;  team-building  is  difficult;  employees  work  only  as  hard  as  they  have  to,  and  are 
susceptible  to  shirking. 

While  any  one  ‘type’  might  seem  extreme.  Hood  (1998)  argues  that  hybrids  often  form  when 
different  types  work  together.  Research  granting  bodies,  such  the  Social  Sciences  and 
Humanities  Research  Council  (SSHRC)  or  the  Natural  Sciences  and  Engineering  Research 
Council  (NSERC),  are  good  examples.  These  funding  bodies  combine  the  peer  group  judgment 
of  egalitarianism  and  the  competition  of  individualism  when  they  strike  expert  committees  to 
determine  which  research  applications  will  receive  grants. 

These  hybrid  forms  can  be  tenuous  arrangements,  however.  When  a  hybrid  system  is  under  stress 
or  fails,  participants  will  revert  to  their  preferred  type.  Because  all  types  have  Achilles’  heals, 
participants  will  blame  the  failings  of  hybrid  governance  structures  on  the  type  with  which  they 
do  not  identify.  Returning  to  the  research  granting  bodies  example,  if  such  bodies  fail,  the 
individualists  are  likely  to  blame  the  lack  of  competition  or  the  egalitarians’  preoccupation  with 
process  over  results.  The  egalitarians,  on  the  other  hand,  might  blame  the  pressure  from  external 
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forces  to  fund  projects  with  some  market  or  consumer  value  rather  than  what  the  ‘expert’ 
community  itself  values. 

Generally  governments  move  between  types  over  time.  Hood  (1998)  notes  that  governments  tend 
towards  one  type;  they  experience  the  failures  to  which  the  type  is  susceptible  and — by  way  of 
responding  to  the  failures — they  shift  to  another  type.  They  employ  the  practices  of  the  second 
type  until  they  experience  the  failures  related  to  this  second  type,  and  then  they  shift  again,  often 
back  to  the  first  type.  For  example,  over-regulation  in  many  Anglo-democracies  in  the  1970s — a 
hierarchical  tendency — resulted  in  the  individualist-type  deregulation  of  New  Public  Management 
reforms  of  the  1980s  and  1990s.  The  inability  of  pay-for-performance  schemes  to  motivate 
public  servants  and  the  corruption  associated  with  contracting-out  many  government  services, 
however,  resulted  in  a  return  to  many  command  and  control  hierarchical  tendencies  in  these 
countries  in  the  21**  Century.  Indeed  it  is  like  a  pendulum;  Cultural  Theorists  would  suggest  these 
types  of  shifts  were  and  are  inevitable. 

Cultural  Theory  has  had  limited  success  when  tested  empirically.  (See,  for  example,  Dake  1991; 
Sjoberg  1997.)  Dake  had  some  success  but  noted  the  correlations  between  culture  and  bias  were 
weak  and  of  limited  predictive  value.  The  typology  is  also  criticized  on  the  grounds  that  the 
categories  are  too  limiting.  Assumptions  about  risk  perception  are  far  more  complex  and 
dynamic  than  the  categories  imply  (Renn  et  al  1992),  and  Cultural  Theory  also  fails  to  take  the 
media  into  account  (Zinn  2004,  15). 

Still,  its  value  should  not  be  underestimated.  Cultural  Theory  has  been  described  as  a 
revolutionary  change  to  risk  perception  (Royal  Society  1992,  1 12).  If  one  accepts  the  argument 
that  regulation  and  integration  determine  one’s  understanding  of  risk  and  ‘good’  governance,  then 
there  is  no  single  metric  for  risk  analysis.  Rather,  people  will  understand  risk  according  to  their 
own  bias,  will  set  up  governance  structures  to  protect  their  preferred  systems  and  will  blame 
failings  of  governance  on  the  shortcoming  of  other  ‘types.’  This  understanding  of  risk  embeds  a 
degree  of  instability  at  the  heart  of  governance,  which  will  be  discussed  further  in  the  next 
section.  Table  A.l,  overleaf,  summarizes  the  discussion  in  Section  II. 


A.3  Using  Cultural  Theory  as  a  Guide  to  Examine  Risk 
Management  Practices 

Approximately  two-thirds  of  the  respondents  in  the  DRDC  survey  consistently  suggest  that  they 
‘sometimes,’  ‘rarely’  or  ‘never’  perform  risk  management.  It  is  not  clear  exactly  what  definition 
of  risk  management  they  are  using  but  it  would  seem  to  be  one  that  refers  only  to  practices 
formally  labeled  ‘risk  management.  ’ 

One  might  conclude  after  reviewing  the  DRDC  MAF  survey  data  that  the  respondents  know  what 
risk  management  is  and  they  are  not  doing  it  very  often.  Ironically,  if  one  examines  the  situation 
through  the  lens  of  Cultural  Theory  one  might  draw  the  opposite  conclusion;  that  in  fact,  the 
respondents  are  engaged  in  considerable  risk  management  but  they  are  not  aware  that  what  they 
are  doing  is  a  form  of  risk  management. 
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Table  A.l:  Cultural  Theory  Summary  Table  (Based  on  Hood,  1998) 


Regulation 


Fatalist 

Hierarchist 

•  Stress:  unpredictability  &  unintended 

•  Stress:  expertise,  forecasting. 

effects 

management 

•  Organisational  design:  rule-bound 

•  Organisational  design:  top/down; 

•  Tactics:  low  trust 

bureaucracy 

•  Blame:  the  ‘fickle  finger’  of  fate 

•  Tactics:  rule-driven 

•  Remedy:  minimal  anticipation,  ad  hoc 

•  Blame:  poor  compliance  with 

responses 

procedures,  lack  of  expertise 

•  Watchword:  ‘resilience’ 

•  Remedy:  more  expertise,  procedures 

•  Weakness:  Unwillingness  to  plan 

•  Watchword:  ‘Steering’ 

ahead  or  take  drastic  measures  in 

•  Weakness:  misplaced  trust  in 

extreme  circumstances 

authority 

•  Vulnerable:  failures  stemming  from 

•  Vulnerable:  dramatic  collapse  of  big 

excessive  inertia  and  passivity 

projects,  sweeping  problems  under 

the  rug 

•  Stress:  individual  potential;  individuals 

•  Stress:  group  and  power  structures 

as  rational 

•  Organisational  design:  flat 

•  Organisational  design:  atomised 

•  Tactics:  face-to-face  accountability 

•  Tactics:  negotiation  &  competition 

•  Blame:  abuse  of  power  by  top  levels. 

•  Blame:  faulty  incentive  structures, 

system  corruption 

lack  of  price  signals 

•  Remedy:  participation,  community 

•  Remedy:  market-like  mechanisms. 

•  Watchword:  ‘community  participation’ 

competition 

•  Weakness:  Unwillingness  to  accept 

•  Watchword:  ‘enlightened  self-interest’ 

higher  authority  to  break  deadlocks 

•  Weakness:  Tendency  to  put  individual 

•  Vulnerable:  failures  stemming  from 

before  collective  benefit 

unresolved  feuds  or  collegiality 

•  Vulnerable:  failures  stemming  from 

degenerating  into  co-existence 

lack  of  cooperation  or  individual 

corruption 

Individualist 

Egalitarian 

Integration 


Cultural  Theorists  would  argue  we  establish  institutions — such  as  DRDC — ^to  manage  specific 
‘risks,’  which,  according  to  this  theory,  means  ‘bads’  or  ‘dangers.’  If  we  accept  this 
understanding  of  risk  management,  we  see  evidence  of  risk  management  in  each  project  or 
program  that  DRDC  runs,  whether  or  not  it  is  formally  identified  as  ‘risk  management.’  Policy 
branches  manage  defence  research-related  policy  risks,  communications  branches  manage 
defence  research-related  communications  risks,  and  so  on.  The  research  projects  in  which  DRDC 
is  engaged  are  themselves  responses  to  risks. 

DRDC  has  responded  to  the  risks  by  employing  an  institutional  design  that  is  home  out  of  its 
values,  sense  of  risk  and  existing  institutional  arrangements.  According  to  the  Theory,  this 
institutional  design  will  inevitably  have  strengths  and  blind  spots.  Cultural  Theorists  would 
further  suggest  that  DRDC’s  response  to  risk  may  in  fact  reinforce  the  blind  spots  rather  than 
challenge  and  correct  them. 

The  following  sections  use  Cultural  Theory  to  diagnose  possible  tensions  in  the  organizational 
culture,  suggest  failures  that  may  emerge  as  a  result  of  the  organizational  design  and  propose 
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ways  of  responding  to  these  failures — either  through  organizational  design  and  management 
practice  or  the  development  of  competencies  and  specific  training. 

For  the  purposes  of  this  analysis,  1  assume  that  DRDC — like  many  large  public  agencies — is 
largely  bureaucratic.  In  Cultural  Theory  terms,  it  would  have  strong  hierarchical  tendencies.  1 
also  assume  that  DRDC  has  highly  specialized  research  teams  that  are  not  especially  hierarchical 
that  exist  as  units  within  the  bureaucratic  framework. 

This  section  is  somewhat  speculative;  it  has  not  been  tested  at  DRDC.  Rather,  the  discussion 
takes  the  insights  of  Cultural  Theory  at  face  value  and  reads  them  into  DRDC,  if  you  will.  In 
many  respects  the  analysis  is  fairly  generic  and  could  be  applied  to  several  public  agencies  that 
share  DRDC’s  chief  characteristics.  As  noted  at  the  beginning  of  this  paper,  the  purpose  of  the 
paper  is  to  use  the  framework  to  stimulate  some  thinking  about  risk  management  at  DRDC. 
Finally,  Cultural  Theory — as  noted  in  the  previous  section  and  like  all  theories — has  its  own 
limitations;  these  limitations  should  be  kept  in  mind  when  reading  the  analysis. 

A.3.1  Hierarchy  and  Egalitarianism:  Holding  Groups  to  Account  in 
Bureaucracies 

Highly  professional  research  teams  situated  within  bureaucracies  suggest  both  egalitarian  and 
hierarchical  characteristics.  Bureaucracies  have  hierarchical  tendencies;  they  lean  towards 
clarifying  roles  and  responsibilities  and  recruiting  specialists,  as  required.  Research  teams, 
however,  can  often  be  egalitarian;  they  are  flat  (little  hierarchy)  and  have  a  relatively  small 
membership  with  a  strong  group  identity  that  is  borne  of  a  commitment  to  the  research  subject 
and  the  team’s  professional  training  rather  than  as  a  commitment  to  the  hierarchical  organization 
itself 

Managing  strong  teams  within  bureaucracies  presents  challenges.  While  bureaucracies  expect 
those  in  subservient  positions  to  account  formally  for  their  work  to  their  superiors,  egalitarian 
communities  resist  outward  (or  upward,  as  the  case  may  be)  accountability.  The  key  for  the 
organization  is  to  hold  the  teams  to  account  for  spending  public  money  without  demoralizing  the 
team  or  losing  the  benefit  of  professional  commitment  to  the  research  subject  by  what  can  be 
interpreted  as  ‘excessive’  bureaucratic  reporting  requirements. 

It  is  important  to  strike  a  balance  between  the  tendencies  of  the  two  types.  Management  might 
insist  on  having  formal  performance  standards,  for  instance,  but  it  might  require  that  each 
research  team  devise  or  have  significant  input  into  the  performance  standards  by  which  the  team 
will  be  held  accountable.  Management  might  also  emphasize  peer  assessments  as  opposed  to  top- 
down  methods  of  evaluation.  If  the  teams  develop  their  own  performance  standards  and  assess 
one  another’s  work,  the  team  will  not  feel  (to  the  same  extent)  its  professionalism  is  being 
undermined  by  authorities  outside  of  the  research  team.  Management,  on  the  other  hand,  will 
have  standards  by  which  they  can  judge  the  research  team  on  a  project  or  year  over  year  basis. 
This  approach  does  risk  ‘collective’  shirking  or  underperforming;  team-based  performance 
awards  can  help  to  mitigate  this  risk. 

A  somewhat  different  but  potentially  useful  example  of  this  practice  occurred  at  universities  in 
the  United  Kingdom  (UK).  The  UK  government  wanted  to  assess  research  productivity  at 
publicly  funded  UK  universities.  At  the  government’s  request,  UK  universities  devised  their  own 
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method  of  evaluating  and  reporting  on  research,  the  Research  Assessment  Exercise  (RAE).  The 
approach  managed  the  competing  pressures  of  holding  universities  to  account  for  their  publicly 
funded  research  while  also  allowing  universities  sufficient  autonomy  to  pursue  their  work  as  best 
they  saw  fit. 

The  RAE  combines  the  rules  of  hierarchy  with  the  inward  accountability  of  groups  and  financial 
motivators  and  competition  of  individualism.  Under  the  RAE,  academic  departments  submit 
publications  to  an  academic  peer  review  panel,  which  grades  and  ranks  their  work.  The  national 
funding  councils  then  use  the  rankings  to  inform  the  allocation  of  each  university.  The  RAE  is 
not  without  its  critics.  Still,  the  method  has  been  cited  as  having  been  successful  at  improving 
aspects  of  research  performance  at  UK  universities.  (For  a  fuller  discussion  of  the  RAE,  see  the 
Roberts  Review  2003.)  For  our  purposes,  the  RAE  highlights  the  capacity  to  improve 
accountability  and  transparency  without  eroding  the  benefits  of  a  strong,  professional  and 
egalitarian  culture. 

This  tension  between  egalitarians  and  hierarchists  suggest  competencies  that  DRDC  may  wish  to 
develop  within  the  agency.  Public  managers  should  consider  a  particular  unit’s  location  on  the 
grid/group  typology,  and  develop  an  awareness  of  the  unit’s  preferred  governance  structures. 
Strong  groups  are  difficult  to  hold  to  account  within  bureaucratic  structures;  in  this  context, 
negotiation  between  parties — rather  than  command  and  control  style — may  be  a  more  successful 
approach  to  developing  performance  standards. 

A.3.2  Fatalism:  The  Complexity  of  Modern  Systems 

Many  failures  occur  because  the  systems  (and  their  interdependencies)  are  not  easily  understood. 
(See  for  instance  Perrow  1999;  Vaughan  1996;  Clarke  2006.)  Renn  (2008)  draws  a  distinction 
between  simple  risks — in  which  cause  and  effect  relationship  are  clear — and  complex  risks — in 
which  causal  links  between  a  multitude  of  potential  cause  agents  and  specific  observed  effects  are 
difficult  to  identify  and  quantify.  Complex  risks  challenge  even  the  most  sophisticated  risk 
management  plans. 

We  often  respond  to  complex  risks  by  attempting  to  model  the  systems.  Up  until  the  1980s  the 
study  of  risk  was  dominated  in  most  Western  countries  by  scientists,  engineers,  economists  and 
decision  analysts.  Arguably,  it  continues  to  be  today.  In  these  communities  formal  modeling — 
employing  statistical  analyses  and  probability  risk  assessments,  for  instance — is  a  very  common 
practice.  These  techniques  potentially  make  opaque  systems  more  transparent.  Within  this  view, 
technical  risk  analyses  are  assumed  to  be  able  to  reveal,  avoid  and/or  modify  the  causal  agents 
associated  with  unwanted  effects. 

Still,  from  a  normative  standpoint  formal  modelling  embeds  key  assumptions  that  potentially 
hinder  its  success.  To  start,  when  we  model  we  assume  complex  technological  systems  are 
accessible  to  detailed  human  comprehension  and  that  a  reductionist  approach  is  the  best  way  to 
understand  the  systems.  This  approach  has  limitations.  Problem  framing,  scientific  conventions 
and  screening  techniques  are  all  subject  to  methodological  biases,  for  instance.  Consider  the 
following.  First,  and  as  noted,  formal  models  frequently  do  not  account  for  all  causal  variables — 
there  are  too  many  and  they  are  not  always  known.  Second,  data  is  often  collected  and  models 
are  built  on  past  experiences,  as  is  usually  the  practice  in  actuarial  science,  for  instance.  These 
models  will  fail  to  predict  new  or  rare  events  because  there  is  insufficient  data  to  satisfy  the 


60 


DRDC  Atlantic  CR  2009-136 


model.  Third  (and  relatedly),  when  data  is  unavailable  for  risk  models — and  it  often  is 
unavailable  when  we  are  exploring  rare  events  (e.g.  acts  of  terrorism) — the  data  is  often 
estimated.  Estimations  embedded  at  several  levels  of  complex  models  will  weaken  the  overall 
validity  of  the  model. 

There  are  also  several  human  aspects  that  are  difficult  to  account  for  in  formal  models.  The 
concept  of  ‘consequence’  is  embedded  in  our  understanding  of  ‘risk’  (that  is,  risk  is  probability 
multiplied  by  consequence).  However,  what  people  perceive  as  ‘unwanted  effects’  or 
‘consequence’  differs,  which  can  undermine  attempts  to  have  a  standard  risk  measurement. 
Willingness  to  tolerate  certain  risks  will  vary  across  individuals,  organizations,  societies  and  time 
periods,  for  instance.  Moreover,  the  interaction  between  human  activities  and  consequences  is 
more  complex  and  perhaps  subtle  than  the  average  probabilities  captured  by  most  risk  analyses. 
Finally,  the  institutional  structure  of  managing  and  controlling  risks  is  prone  to  organizational 
failure,  which  may  increase  actual  risks  (Jaeger  et  al.  2001,  86).  For  instance,  risk  data  can  be 
difficult  to  collect  because  individuals  who  are  potentially  responsible  for  operational  failures 
may  not  wish  to  disclose  their  failures  to  others,  and  especially  not  to  their  superiors.  In  short, 
they  tailor  the  information  or  withhold  it  altogether. 

Rather  than  a  perfect  formula  or  model,  risk  management  requires  a  measure  of  deliberation  and 
judgement,  balance  and  indeed,  good  luck.  When  organizations  expend  resources  on  certain 
risks,  other  risks  are  inevitably  neglected.  Moreover,  the  very  act  of  intervening  with  a  risk 
management  strategy  is  a  risk  in  itself,  and  therefore  risk  management  creates  risks.  Sunstein 
noted  when  asked  about  the  precautionary  principle  and  guarding  against  risks  to  critical 
infrastructure: 

The  problem  is  that  the  precautionary  principle  forbids  the  very  steps  that  it  requires, 
because  precautions  themselves  create  risks.  If  the  precautionary  principle  says  that  we 
should  build  a  margin  of  safety  into  all  decisions,  there’s  a  problem:  risks  are  on  all  sides, 
and  so  it’s  hard  to  have  a  margin  of  safety  against  all  risks!  If  you  stay  home,  you  create 
risks;  so  too,  if  you  go  to  work;  so  too,  if  you  exercise;  so  too,  if  you  don’t  exercise.  The 
Iraq  War  was  plausibly  defended  partly  on  precautionary  grounds;  it  was  plausibly 
criticized  on  those  same  grounds.  Aggressive  steps  to  prevent  climate  change  are 
plausibly  defended  on  precautionary  grounds,  but  such  steps  might  well  violate  the 
precautionary  principle  because  they  too  create  risks.  We  can  imagine  steps  involving 
critical  infrastructure  that  are  required  by  the  principle  -  but  that  also  offend  it,  because 
they  impose  costs  and  create  risks  of  their  own  (Sunstein,  as  cited  in  Quigley  2008). 

Decision-makers  must  allocate  limited  resources  to  manage  unlimited  risks.  They  must  therefore 
strike  a  balance  in  the  resources  they  allocate  to  managing  different  risks;  they  must  also 
recognize  that  even  the  best  intended  risk  management  intervention  can  go  awry  due  to  errors  in 
judgements  or  due  to  the  risks  one  generates  by  one’s  own  intervention  strategy. 

That  risk  is  not  easily  identified  or  controlled  has  five  implications  for  risk  management 
competencies. 

First,  complex  risks  have  a  multitude  of  potential  causes  and  are  therefore  difficult  to  identify  and 
quantify.  Public  managers  should  be  aware  of  the  different  quantitative  and  qualitative  tools  and 
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processes  available  for  assessing  risks.  At  the  same  time,  public  managers  should  always 
maintain  a  level  of  scepticism  about  their  ability  to  understand  and  control  risk  fully. 

Second,  public  managers  should  be  able  to  prioritize  risks,  yes,  but  also  to  develop  a  self- 
awareness  of  their  actions.  They  should  monitor  not  only  the  progress  on  the  target  risk  source 
but  also  consider  the  unintended  consequences  of  their  behaviours.  Training  should  help  public 
managers  develop  this  sense  of  awareness  and  allow  them  to  practise  making  judgements  about 
when  and  how  much  to  invest  in  managing  an  unlimited  list  of  risks. 

Third,  because  cause  and  effect  relationships  are  problematic,  organizations  on  occasion  will  have 
to  respond  to  events  that  cannot  be  anticipated.  Public  managers  should  therefore  develop  a 
capacity  to  respond  ‘on  the  fly’;  to  get  up  and  running  after  the  organization  has  been  dealt  a 
setback.  The  key  word  here  is  ‘resilience.’ 

Fourth,  it  is  important  to  consider  the  consequences  of  failures  and  extend  that  analysis  to  risk 
management  planning.  Often  public  agencies  are  criticized  for  over-reacting  /  over-regulating 
risks.  Cost-Benefit  Analyses  (CBAs)  provide  useful  guides  in  determining  appropriate  levels  of 
regulation.  CBAs  aim  to  create  a  consistent  and  logically  sound  model  of  a  person’s  or 
institution’s  knowledge  and  preference  structure  (Jaeger  et  al.  2001,  79). 

Nevertheless,  many  social  and  organizational  dynamics  constrain  our  ability  to  develop  accurate 
CBAs.  CBAs  require  that  participants  engage  in  multiple  acts  of  conversion,  assigning  values  to 
such  entities  as  human  lives,  human  morbidity  and  a  range  of  harms  to  the  environment.  The 
concept  assumes  ‘positives’  and  ‘negatives’  can  be  articulated,  compared  and  judged  in  a  single 
measurement:  usually  dollars.  Such  conversions  are  difficult,  potentially  unstable  and  rarely  done 
consistently  within  organizations. 

Moreover,  there  may  be  times  when  CBAs  may  be  unhelpful,  and  more  precautionary  approaches 
may  be  more  appropriate.  Sunstein — normally  a  sceptic  of  precautionary  approaches — suggests 
that  when  we  consider  ‘irreversible  harms’  or  ‘catastrophic  disasters’  for  instance,  we  may  wish 
to  proceed  with  more  caution,  considering  worst-case  scenarios,  however  unlikely  they  may  be 
(Sunstein  2005;  2007). 

In  sum,  public  managers  should  practise  the  application  of  these  risk  management  concepts — 
CBAs,  worst  case  scenarios  and  precautionary  approaches — and  know  how  and  when  to  use 
them. 

Fifth,  the  complexity  of  the  risk  can  make  it  difficult  to  assign  responsibility  to  individuals.  This 
can  be  particularly  frustrating  in  a  bureaucratic  context.  In  certain  cases,  increasing  the  level  of 
transparency  and  the  frequency  of  reporting  can  be  a  helpful  alternative.  Again,  public  managers 
must  determine  what  level  of  reporting  is  appropriate,  knowing  that  regular  reporting  can  be  a 
drain  on  resources  and  potentially  undermine  staff  morale. 

A.3.3  Individualism:  Deregulation  as  a  Springboard  to  Innovation 

Risk  and  innovation  might  usefully  be  thought  of  together.  If  we  think  of  risk  as  ‘uncertainty’ — 
that  something  potentially  ‘good’  may  occur  as  well  as  something  ‘bad’  or  ‘dangerous’ — ^then 
there  may  be  opportunities  to  exploit  conditions  for  individual  or  collective  gain.  Public 
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agencies  might  therefore  promote  a  level  of  policy  entrepreneurship,  which  might  encourage  or 
celebrate  those  public  servants  who  can  negotiate  and  exploit  the  complex  social  and  institutional 
context  to  achieve  in  the  face  of  uncertainty  an  institutional  goal. 

New  and  emerging  technologies  present  one  such  example  for  policy  entrepreneurs.  As  many  in 
the  public  service  retire,  considerable  institutional  memory  will  exit.  At  the  same  time  the  new 
generation  entering  the  public  service  is  perhaps  more  comfortable  with  leveraging  new  social 
technologies  that  facilitate  the  networking  with  universities,  think  tanks,  and  on-line  and  virtual 
communities  that  may  help  to  fdl  the  gap  that  those  heading  towards  retirement  will  leave  behind. 
Indeed,  policy  entrepreneurs  can  exploit  new  technologies  to  tap  into  committed  groups  and 
subject  matter  experts  that  do  not  necessarily  see  themselves  as  traditional  ‘stakeholders’  but  can 
help  inform  public  policy  debates. 

Cultural  Theorists  would  argue  that  in  order  to  achieve  a  setting  that  is  conducive  to  more 
creative  outcomes  public  institutions  should  deregulate.  This  deregulation  can  include  facilitating 
access  among  public  servants  to  new  and  emerging  technologies,  including  the  removal  of 
unnecessary  firewalls,  for  instance.  This  approach  would  also  emphasize  more  competition  and 
market-like  mechanisms  as  a  way  of  encouraging  entrepreneurialism,  such  as  the  RAE  example 
cited  above,  in  which  research  teams  compete  against  each  other  for  financial  rewards.  This 
arrangement  is  potentially  less  stable  and  has  the  potential  for  more  conflict,  but  it  can  be  more 
dynamic  and  adaptive,  also. 

These  ends  are  not  easily  achieved  in  traditional  bureaucratic  structures  or  strong  teams.  Public 
institutions  would  adopt  practices  that  align  public  managers’  private  interests  with  the  public 
good  through  pay-for-performance  contracts  among  public  servants,  for  instance.  Governments 
could  also  partner  with  more  flexible  and  deregulated  delivery  agents,  especially  in  the  private 
sector,  though  contract  management  would  then  become  a  risk  the  organization  would  have  to 
manage  more  often. 

Finally,  benefits  designed  to  appeal  to  the  individuals  alone  can  potentially  undermine  group 
cohesion  within  research  teams.  Performance  awards  might  therefore  work  at  the  group  level 
rather  than  the  individual  level.  Alternatively,  individual  awards  can  be  determined  by  the 
research  group  itself  in  order  to  maximize  face-to-face  accountability. 

A.4  Conclusion  and  Proposed  Training 

There  are  several  theoretical  frameworks  one  could  employ  to  analyze  risk  management  in 
organizations.  Cultural  Theory  is  one  way.  High  Reliability  Organizations  (HROs)'^^  and 
Normal  Accidents  Theory'"*”  are  recognized  alternatives. 

Cultural  Theory  can  be  a  useful  heuristic  device  that  can  help  guide  our  reasoning  about  risk 
management  and  its  potential  blindspots.  It  can  help  us  to  anticipate  failings,  and  teach  us  how  to 
avoid  them  or  at  least  reduce  their  negative  impacts.  As  different  ‘types’  understand  risk 
differently,  however,  solutions  from  Cultural  Theory  can  often  be  unstable. 


For  a  discussion  of  HRO,  see,  for  example,  Laporte  (1996)  and  Laporte  and  Consolini  (1991). 
For  a  discussion  of  NAT,  see,  for  example,  Perrow  (1999),  Vaughan  (1996)  and  Clarke  (2006). 
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The  purpose  of  the  paper  is  to  stimulate  some  thinking  about  risk  management  practices.  With 
this  in  mind,  1  took  the  insights  of  Cultural  Theory  at  face  value  and  read  them  into  DRDC,  if  you 
will.  1  assume,  for  instance,  that  DRDC’s  is  largely  bureaucratic  and  has  strong  hierarchical 
tendencies,  that  research  teams  are  essentially  flat  (i.e.  not  hierarchical),  and  that  these  two 
arrangements  likely  exemplify  a  clash  between  egalitarianism  (research  teams)  and  hierarchy  (the 
DRDC  bureaucracy  itself).  This  analysis  is  somewhat  speculative;  it  has  not  been  tested  at 
DRDC. 

The  paper  suggests  that  managers  within  the  organization  can  reduce  the  negative  tensions  that 
result  from  the  clash  of  egalitarianism  and  hierarchy  by  emphasizing  negotiation  and  cooperation 
between  management  and  the  research  teams  rather  than  employing  a  top-down,  command  and 
control  style.  It  also  suggests  that  the  organization  might  stimulate  innovation  through 
deregulation,  partnering  with  flexible  organizations  and  emphasizing  individual  and  team 
rewards. 

Finally,  the  paper  underscores  that  there  is  no  perfect  approach  to  managing  risk  and  uncertainty. 
There  are  many  risks,  and  resources  available  to  manage  them  are  limited.  Risk  management 
strategies  themselves  can  generate  risks.  Rather  than  eliminate  risks,  we  often  decrease  certain 
risks  by  increasing  other  risks.  Ultimately  we  seek  to  strike  a  balance;  to  reduce  risks  to 
acceptable  levels  of  risk  exposure.  Understanding  the  context  in  which  risk  emerges  and  is 
interpreted  is  crucial. 

Building  on  the  themes  in  this  paper,  a  related  training  plan  for  staff  at  DRDC  would  include: 

•  familiarizing  staff  with  a  broad  understanding  of  risk,  risk  assessments  and  risk 
management,  drawing  on  diverse  literature  and  methods; 

•  introducing  staff  to  the  challenges  of  and  limitations  to  measuring  risk,  of  determining  the 
probability  and  consequence  of  events,  for  instance; 

•  helping  staff  develop  a  greater  self-awareness  of  their  risk  management  plans  (e.g.  when 
they  are  engaged  in  risk  management  activities;  the  unintended  consequences  of  their  risk 
management  interventions); 

•  introducing  staff  to  tools,  processes  and  heuristic  devices  that  do  not  necessarily  give 
definitive  answers  about  risk  management  but  can  help  guide  public  managers’  reasoning 
when  they  make  risk  management  intervention  (e.g.,  cost-benefit  analyses;  precautionary 
approaches;  multi-disciplinary  methodologies); 

•  incentivizing  entrepreneurial  behaviour  to  exploit  risk  and  uncertainty  for  institutional 
advantage. 
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